For automation or bulk exports, PowerShell is the preferred tool. You must use the Get-AzKeyVaultSecret command because the private key resides in the secret backing the certificate. Export certificates from Azure Key Vault | Microsoft Learn

Navigate to your and select Certificates from the left-hand menu. Click on the name of the certificate you wish to download. Select the Current Version of the certificate. Click the Download in PFX/PEM format button.

: The certificate must have been created or imported with an Exportable private key. If the policy was set to "non-exportable," the private key cannot be retrieved.

The Azure Portal provides a direct UI option for manual downloads.