I'm running Airflow 1.10.6 on Google Cloud Composer with Python 3. To initiate a dataflow job from the composer environment I need to have the PyPi package google-cloud-datastore. When I try to add this package through either the interface or cloud shell, the build fails. It raises the following error:
UPDATE operation on this environment failed 1 hour ago with the following error message:
Failed to install PyPI packages.
If I check the logs of Google Build for the Kubernetes Engine I see the following error:
I 2020-03-04T14:36:23.939075607Z [0mgoogle-cloud-datastore 1.11.0 has requirement google-cloud-core<2.0dev,>=1.0.3, but you have google-cloud-core 0.29.1.
I 2020-03-04T14:36:23.939669242Z google-cloud-datastore 1.11.0 has requirement google-api-core[grpc]<2.0.0dev,>=1.14.0, but you have google-api-core 1.8.1.
So google-cloud-datastore requires google-cloud-core>=1.0.3. I tried to explicitly set google-cloud-core to 1.0.3, but then I get the following error:
I 2020-03-05T08:13:18.539300693Z [0mgoogle-cloud-logging 1.9.1 has requirement google-cloud-core<0.30dev,>=0.29.0, but you have google-cloud-core 1.0.3.
I 2020-03-05T08:13:18.539878367Z google-cloud-bigtable 0.32.0 has requirement google-cloud-core<0.30dev,>=0.29.0, but you have google-cloud-core 1.0.3.
I 2020-03-05T08:13:18.540308937Z google-cloud-bigquery 1.8.1 has requirement google-cloud-core<0.30dev,>=0.29.0, but you have google-cloud-core 1.0.3.
So the ranges of versions required for the packages are not overlapping. Does anyone know if this can be fixed? Please let me know.
There is a compatibility issue with google-cloud-datastore in the main container of Composer, and that the latest version of this package could not be installed. The Composer product team has confirmed the issue and is working on it.
Attempting to install an older version of google-cloud-datastore may be a workaround before the root cause is fixed. I have tried with 1.7.4 version and got positive result:
I hope it helps.
In case you (or someone else reading this) needs to use the latest version of google-cloud-datastore (==1.11.0), I was able to install it in my environment (composer-1.9.1-airflow-1.10.6) by adding recent versions of the following libraries to the dependencies too:
googleapis-common-protos==1.6.0
google-cloud-spanner==1.14.0
google-cloud-bigtable==1.2.1
google-cloud-logging==1.14.0
google-cloud-bigquery==1.22
Related
I'm trying to install azureml-sdk in a Mac M1 with Python 3.9.15 and facing the following problems:
When I pip install azureml-sdk, it installs an old version (azureml==1.0.23)
If I run:
from azureml.core import Workspace
ws = Workspace.from_config()
I get the following error:
AttributeError: 'BasicTokenAuthentication' object has no attribute 'get_token'
And if I try to install the latest azureml-sdk version with
pip install azureml-sdk==1.48.0
I get the following error:
ERROR: Cannot install azureml-dataset-runtime because these package versions have conflicting dependencies.
The conflict is caused by:
azureml-dataprep 4.8.3 depends on azureml-dataprep-native<39.0.0 and >=38.0.0
azureml-dataprep 4.8.2 depends on azureml-dataprep-native<39.0.0 and >=38.0.0
azureml-dataprep 4.8.1 depends on azureml-dataprep-native<39.0.0 and >=38.0.0
azureml-dataprep 4.8.0 depends on azureml-dataprep-native<39.0.0 and >=38.0.0
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
Any ideas? Thanks in advance
Regarding the first error you cited, this error is typically experienced when the provided login credentials for the Azure Machine Learning workspace are incorrect.
Verify the access rights for the workspace on the service principal you are using for authentication.
Make sure the configuration file you use for authentication is formatted correctly and has all the necessary elements.
Use the newly created service principal and configuration file to authenticate.
The second error is because these package versions have conflicting dependencies, typically experienced when there is a conflict between the dependencies of the installed packages.
You might attempt the following solutions to troubleshoot this problem:
Try downgrading or upgrading the conflicting packages' versions until you find one that works with the Azure Machine Learning SDK.
To avoid installing the dependencies, try installing the Azure Machine Learning SDK using the —no-deps parameter.
If the issue still exists, you can try deleting and then reinstalling every package linked to Azure Machine Learning.
I'm trying to use install snowflake-connector-python to a Cloud Composer environment but keep receiving an error that pops up in the UI, "Failed to install PyPI Packages. Check the Cloud Build log for details". The build log doesn't seem overly helpful and is very long, so I'll just show some of the notable things here.
This is a warning that shows after installing snowflake-connector-python and its dependencies.
Installing collected packages: pycryptodomex, asn1crypto, oscrypto, jmespath, botocore, s3transfer, boto3, isodate, msrest, azure-core, azure-storage-blob, azure-common, snowflake-connector-python
Successfully installed asn1crypto-1.4.0 azure-common-1.1.25 azure-core-1.8.1 azure-storage-blob-12.5.0 boto3-1.14.63 botocore-1.17.63 isodate-0.6.0 jmespath-0.10.0 msrest-0.6.19 oscrypto-1.2.1 pycryptodomex-3.9.8 s3transfer-0.3.3 snowflake-connector-python-2.3.2
+ [[ -z fail ]]
+ python3 -m pipdeptree --warn fail
Warning!!! Possibly conflicting dependencies found:
* botocore==1.17.63
- docutils [required: >=0.10,<0.16, installed: 0.16]
This is the final error notification.
The command '/bin/sh -c bash installer.sh $COMPOSER_PYTHON_VERSION fail' returned a non-zero code: 1
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 1
I'm running the image composer-1.12.1-airflow-1.10.10 in zone us-central1-c and is using python 3. I've tried installing different versions of the package with no luck, and have tried doing it via the UI and via gcloud. Any help as to the root of this problem would be much appreciated!
I tried trying to install older versions of snowflake-connector-python but that didn't work.
What did end up working was adding docutils==0.15 to the list of packages to install. This fixed the dependency conflict warning and led to a successful build.
Previously, I've received dependency conflict warnings that led to successful builds. But for this case, I actually needed to resolve that conflict.
The answer from nate did not work for me, the PyPi package install succeeds in CloudBuild but then the Web Server fails to update and rolls back installing the packages.
In order to get this working we had to downgrade our Composer to the September release which comes with docutils==0.15.2.
The image we used was composer-1.12.1-airflow-1.10.9, it seems that any image in airflow 1.10.10 or above has this issue as they come with docutils==0.16 as standard. You can see which version of docutils is in an image by using this link and expanding the PyPi packages column.
I am getting the following error while duilding tic-tac-toe game with react.
"./src/index.css (./node_modules/css-loader/dist/cjs.js??ref--6-oneOf-3-1!./node_modules/postcss-loader/src??postcss!./src/index.css)
Error: Package exports for 'G:\Nodejs\projectReact\tic-tac-toe\node_modules\colorette' do not define a valid '.' target"
I faced a similar issue when I had the latest colorette version installed i.e. 1.2.1. As per my understanding, this version of colorette requires Node 14+ while I have Node 13.1. Therefore there are two options to resolve this, either upgrade Node to 14+ version of downgrade colorette to its previous version 1.2.0.
I have not tried upgrading Node as it will effect my other projects so I have just downgraded colorette to 1.2.0 and it works fine for my case by using following commands.
To uninstall colorette:
npm uninstall colorette
To install colorette version 1.2.0:
npm install colorette#1.2.0
Had the same problem when using node v 13.1.x upgrade it to 14.x helped so far using command :
sudo n latest
I am trying to install Openstack using DevStack. After correctly creating the user stack and running ./stack.sh, the script starts to install the required dependencies until it hits Obtaining file:///opt/stack/cinder. Then I obtain the following errors:
Exception:
ContextualVersionConflict: (grpcio 1.21.1 (/usr/local/lib/python2.7/dist-packages), Requirement.parse('grpcio<1.16.0,>=1.0.0'), set(['tooz']))
I understand that there is a conflict in grpcio, but I don't know how to solve it. I already unsuccessfully tried to install a specific version between 1.0.0 and 1.16.0.
The Devstack version I am using is the latest from the git master.
I think I solved the problem myself.
I had the tooz package installed with version 1.62 which was causing the issue.
After removing that package the installation completed.
I follow the instructions to install meteor 1.6 on Windows(10) as stated on Meteor.com (choco install meteor) and it's installed a meteor v0.0.2 ?
When I try to run any meteor command I get: bash: meteor: command not found
and when I try choco upgrade meteor I get:
$ choco upgrade meteor
Chocolatey v0.10.8
Upgrading the following packages:
meteor
By upgrading you accept licenses for the packages.
meteor v0.0.2 is the latest version available based on your source(s).
Chocolatey upgraded 0/1 packages.
See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
I tried choco uninstall meteor and then reinstall choco instlal meteor but same issue. Searching around online a little I found this page:
https://chocolatey.org/packages/meteor
I do have meteor installed and running on version 1.6.0.1 on Window 10's Ubuntu bash which was installed via curl https://install.meteor.com/ | sh
Following on from above post https://stackoverflow.com/a/47967277/642729 by Gary Ewan Park. The below comment on https://chocolatey.org/packages/meteor clarifies the Chocolately/Meteor versioning
The current version of Meteor is 1.6;
is this the version in the 0.0.2 Chocolatey package?
[Garrie Powers • 2 months ago]
The version number of the Chocolatey package is the version of the installer.
Meteor is unique in the way it works in that all versions
are capable of springboarding to other versions.
Therefore, in the same way as the Unix installer installs the latest version,
the Chocolatey installer will always install the latest version.
So at this exact moment, the answer to your question is "yes"!
[Jesse Rosenberger Garrie Powers • 2 months ago]
The best advice would be to reach out to the maintainers of that package of chocolatey.org. This can be done by clicking on the "Contact Maintainers" link of this page:
https://chocolatey.org/packages/meteor
It would seem that the creators of the application, also own the Chocolatey package, so hopefully they will get back to you.
From the official installer github:
The version of this Meteor installer is not to be confused with Meteor itself. Meteor, once installed, will always "springboard" to (download, install and run) the correct version of Meteor necessary for the application being executed.
...
When necessary, specific versions of Meteor can be installed using
Chocoloatey's --params argument which will download that specific
version from Meteor's installation server. For example, to install
Meteor 1.5.4.4:
C:> choco install meteor --params="'/RELEASE:1.6.0.1'"
Note: Prior to
Meteor 1.6, 64-bit versions were not available. Therefore, in order to
install versions prior to Meteor 1.6, you'll also need to pass
Chocolatey's --x86 option when running choco install on 64-bit Windows
platforms. For example:
C:> choco install meteor --x86 --params="'/RELEASE:1.5.4.4'"