我已经安装了气流,我写了一个DAG来将
MySQL数据与BigQuery集成.
当我运行python脚本时,我收到以下错误:
ImportError: cannot import name GbqConnector
我按照指示将大熊猫降级为旧版本.当我这样做时,我得到了一个不同的错误:
ImportError: cannot import name _test_google_api_imports
编辑:来自的建议
x97Core工作.
我现在有一个不同的问题.我收到以下错误:
/usr/local/lib/python2.7/dist-packages/airflow/models.py:1927: PendingDeprecationWarning: Invalid arguments were passed to MySqlToGoogleCloudStorageOperator. Support for passing such arguments will be dropped in Airflow 2.0. Invalid arguments were:
*args: ()
**kwargs: {‘google_cloud_storage_connn_id’: ‘podioGCPConnection’}
category=PendingDeprecationWarning/usr/local/lib/python2.7/dist-packages/airflow/models.py:1927: PendingDeprecationWarning: Invalid arguments were passed to GoogleCloudStorageToBigQueryOperator. Support for passing such arguments will be dropped in Airflow 2.0. Invalid arguments were:
*args: ()
**kwargs: {‘project_id’: ‘podio-data’}
category=PendingDeprecationWarning
根据这个链接,问题在于气流与python2和pythn3的兼容性.我已经尝试在两者上运行代码,但同样的错误仍然出现.
Airflow mysql to gcp Dag error
有谁知道这是否有解决方案?
最佳答案 刚遇到这个问题.
降级大熊猫版本似乎有效(在Airflow v1.8.0上测试):
pip install pandas==0.18.1
有关详细信息:https://issues.apache.org/jira/browse/AIRFLOW-1179
或者,如果您使用的是Airflow 1.8.2或更高版本.
pip install pandas-gbq