Is it possible to load the biconomy web3auth component in _app.tsx?

Yes, it is possible. You can import the the component like this example

https://github.com/bcnmy/sdk-examples/blob/master/nextjs-biconomy-web3Auth/pages/index.tsx