Repository navigation
Invalid configuration value detected for fs.azure.account.key - after update #75
Description
Hi,
After updating this api, I started facing errors in Azure.
My queries before were running fine using the 'utils' package from DBSQLClient, but now, it is giving this error:
Error running query: Failure to initialize configuration Invalid configuration value detected for fs.azure.account.key
From my research, this error is related to authentication in Azure's File Storage.
If this is the case, how can I pass my credentials to correctly execute the queries?
If not, how can I solve this problem?
Thanks.
OperationStateError: The operation failed due to an error
at OperationStatusHelper.<anonymous> (C:\Users\adrianofe\Documents\incident-handling\node_modules\@databricks\sql\dist\DBSQLOperation\OperationStatusHelper.js:92:27)
at Generator.next (<anonymous>)
at fulfilled (C:\Users\adrianofe\Documents\incident-handling\node_modules\@databricks\sql\dist\DBSQLOperation\OperationStatusHelper.js:5:58)
at processTicksAndRejections (internal/process/task_queues.js:95:5) {
response: {
status: {
statusCode: 0,
infoMessages: null,
sqlState: null,
errorCode: null,
errorMessage: null
},
operationState: 5,
sqlState: '42000',
errorCode: 0,
errorMessage: 'org.apache.hive.service.cli.HiveSQLException: Error running query: Failure to initialize configurationInvalid configuration value detected for fs.azure.account.key\n' +
'\tat org.apache.spark.sql.hive.thriftserver.HiveThriftServerErrors$.runningQueryError(HiveThriftServerErrors.scala:53)\n' +
'\tat org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:435)\n' +
'\tat org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.$anonfun$run$2(SparkExecuteStatementOperation.scala:257)\n' +
'\tat scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)\n' +
'\tat org.apache.spark.sql.hive.thriftserver.ThriftLocalProperties.withLocalProperties(ThriftLocalProperties.scala:123)\n' +
'\tat org.apache.spark.sql.hive.thriftserver.ThriftLocalProperties.withLocalProperties$(ThriftLocalProperties.scala:48)\n' +
'\tat org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withLocalProperties(SparkExecuteStatementOperation.scala:52)\n' +
'\tat org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:235)\n' +
'\tat org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:220)\n' +
'\tat java.security.AccessController.doPrivileged(Native Method)\n' +
'\tat javax.security.auth.Subject.doAs(Subject.java:422)\n' +
Metadata
Metadata
Assignees
Labels
No labels
Activity
kravets-levko commentedon Oct 14, 2022
Hi @adrianofewerling! I don't think this error is related to this library. Can you try to roll back to 0.1.8-beta.1 and run the same query against the same warehouse to see if error is still there or not?
adrianofewerling commentedon Oct 14, 2022
Hi @kravets-levko, thanks for your reply
I did what you asked, however, the error was the same.
Recently, the cluster admin changed the policies and I believe this caused the change in the authentication type.
Can you tell me if it is possible to send the ABFS (Azure Blob File Storage) authentication along?
If yes, how can I do it?
I haven't found a way to do this.
Thanks
divyavanmahajan commentedon Feb 19, 2023
Hi
If you are connecting to a SQL Warehouse cluster - set the following in the configuration in SQL Admin. You
Alternatively, switch to using Unity catalog - and query the Unity catalog tables.
kravets-levko commentedon Jan 18, 2024
Hi @divyavanmahajan! I want to ask you to upgrade to latest library version and check it this issue is still relevant. Please let me know (or just close the issue if it no longer reproduces). Thanks!
divyavanmahajan commentedon Jan 19, 2024
Hi @kravets-levko @adrianofewerling - we can close this issue