No single party has access to the SGX enclave address. Only the enclave itself can construct/sign a transaction using that key when it receives approvals from 6 of 8 Bridge Nodes. In this sense, the enclave here is functioning as a cross-chain smart contract.
For a more in-depth read on how the Avalanche Bridge™ works, please read the following articles.
For any additional questions, please visit our knowledge base or contact a support team member via the chat button at support.avax.network.