I trying to make SqlSensor to work with Oracle database, I've installed all the required provider and successfully tested the connection. When I run SqlSensor I got this error message
ERROR - Failed to execute job 32 for task check_exec_date (The connection type is not supported by SqlSensor. The associated hook should be a subclass of `DbApiHook`. Got OracleHook; 419)
I'm running Apache Airflow version 2.3.3 and installed Oracle provider apache-airflow-providers-oracle version 3.2.0
TL;DR:
You are probably importing the sensor as:
from airflow.sensors import SqlSensor
Which cause the issue.
If you will import as
from airflow.providers.common.sql.sensors import SqlSensors
It will work.
Full Details:
There is a bug in apache-airflow-providers-common-sql==1.0.0 which causes
from airflow.sensors import SqlSensor not to work properly.
The bug is fixed in https://github.com/apache/airflow/pull/25293
thus upgrading to apache-airflow-providers-common-sql>1.0.0 will allow also the old import style.
Regardless of the bug, you should use
from airflow.providers.common.sql.sensors import SqlSensors
as from airflow.sensors import SqlSensor is deprecated.
Related
i am getting this message
import {defineConfig} from 'sanity'or its corresponding type declaration
import {deskTool} from 'sanity/desk' or its corresponding type declaration
import {visionTool} from '#sanity/vision' --cannot find module 'sanity/vision' or its corresponding type declaration
tried to run sanity but im getting this above message
I am trying to get the time scale (x-axis) using chartjs-adapter-date-fns adapter and getting this error:
Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: No "exports" main defined in C:\chart\node_modules\chart.js\package.json when using "import 'chartjs-adapter-date-fns" in Next.js'.
This is an issue that currently exists with the chartjs-adapter-date-fns adapter in combination with chart.js#4.0.1. The Luxon adapter has already released a fix for this issue which makes it work with chart.js#4.0.1.
Until it is fixed, you could use the Luxon adapter instead:
npm install luxon chartjs-adapter-luxon --save
import {Chart} from 'chart.js';
import 'chartjs-adapter-luxon';
I am trying to deploy an Airfow DAG to MWAA.
My requirements.txt:
apache-airflow[amazon] == 3.2.0
I import EcsOperator like this:
from airflow.contrib.operators.ecs_operator import EcsOperator
However, I get this error:
Broken DAG: [/usr/local/airflow/dags/mydag.py] Traceback (most recent call last):
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/usr/local/airflow/dags/mydag.py", line 4, in <module>
from airflow.contrib.operators.ecs_operator import EcsOperator
ImportError: cannot import name 'EcsOperator' from 'airflow.contrib.operators.ecs_operator' (/usr/local/lib/python3.7/site-packages/airflow/contrib/operators/ecs_operator.py)
What am I doing wrong here?
What am I doing wrong here?
You might be referencing a different version (1.10.12?) of the Airflow documentation.
airflow.contrib.operators.ecs_operator (1.10.12)
The documentation for 3.2.0 is here. You can import the EcsOperator like this:
from airflow.providers.amazon.aws.operators.ecs import EcsOperator
airflow.providers.amazon.aws.operators.ecs (3.2.0)
The correct requirements.txt:
(empty)
And the correct import:
from airflow.providers.amazon.aws.operators.ecs import ECSOperator
Note the casing!
There are several issues here so I'll compile a detailed answer since privious answers didn't cover all of them.
First, the updated import path (provider release 3.2.0) is:
from airflow.providers.amazon.aws.operators.ecs import EcsOperator
The reason this doesn't work for you is because you install the provider with extras as:
apache-airflow[amazon]
as explained in the provider extra docs when installing provider in that manner you get the provider version which was released at the time of the Airflow version that you are using. Thus you are not guaranteed to get the updated provider version. So in case you are using Airflow 2.2.4 (latest at the time of writing this answer) you will get Amazon provider version 3.0.0 which is not the most recent one.
To get updated provider you should install it as:
pip install apache-airflow-providers-amazon
if you like to pick a specific version then:
pip install apache-airflow-providers-amazon==3.2.0
Please note that you should always install from constraint files provided by Airflow. Example:
pip install "apache-airflow-providers-amazon" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.7.txt"
Note that the provider is referring to constraints-main which constantly updated and not to constraints-2.2.4 or any other specific Airflow version.
You can read more about it in the doc about Installation and upgrading of Airflow providers separately.
I'm trying to run a mixture of SQL and Java migrations via Maven based on the example from Axel Fontaine here: http://www.methodsandtools.com/tools/flyway.php
Basically I am trying to execute several SQL migrations, followed by a java migration (to load BLOBS into a table), then followed by another SQL migration.
The first set of SQL migrations run fine. If I specify a file extension of .java for the Java migration, it gets ignored. If I specify a file extension of .sql for the Java migration, it gets run in the correct sequence, but I get the following error:
[ERROR] com.googlecode.flyway.core.api.FlywayException: Error
executing statement at line 1: package db.migration [ERROR] Caused by
org.postgresql.util.PSQLException: ERROR: syntax error at or near
"package" Position: 1 [ERROR]
com.googlecode.flyway.core.api.FlywayException: Migration of schema
"test" to version 1.0.0106 failed! Changes successfully rolled back.
Here is the head of my Java migration file:
package db.migration;
import com.googlecode.flyway.core.api.migration.jdbc.JdbcMigration;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.io.File;
Any ideas as to what I'm doing wrong?
Okay, I finally figured out what was going on. While Flyway allows version numbers that contain "." in the name (ex. V1.0.0000_filename), apparently it is not supported for Java migration class names. I changed the class name to use "" instead of "." (V1_0_1000_filename) and that allowed me to get past the original error.
I've just migrated from the Python 2.5 to the 2.7 runtime, and most of server I've written runs fine. However I'm occassionaly seeing this odd stack trace (I've hacked it down for brevity):
ERROR 2013-04-23 10:40:15,598 wsgi.py:235]
Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/jinja2-2.6/jinja2/environment.py", line 894, in render
return self.environment.handle_exception(exc_info, True)
File "templates/querystart.html", line 30, in top-level template code
{% for session in sessions %}
ImportError: No module named _sqlite3
I've not imported sqlite3 myself, and I don't rely on it. The code that's calling it is pretty simple (perhaps bordering on the dumb side of things):
class UserIdQuery(BaseHandler):
def get(self):
sessionQuery=Session.all().order("userid")
template_values = {
'sessions': sessionQuery,
}
self.render_template('querystart.html',**template_values)
The above simply extends the below (taken from a very helpful migration tutorial)
class BaseHandler(webapp2.RequestHandler):
#webapp2.cached_property
def jinja2(self):
return jinja2.get_jinja2(app=self.app)
def render_template(self, filename, **template_args):
self.response.write(self.jinja2.render_template(filename, **template_args))
Anyone know what might be triggering the import error? Thanks,
Appengine uses sqlite for the datastore, my bet is your python2.7 installed on OSX is missing the sqlite binary library. Do a quick test, start a python interpreter (outside of appengine) and try to import sqlite3
$ python
Python 2.7.3 (default, Sep 26 2012, 21:53:58)
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>>
If that fails you know you have an incomplete 2.7 runtime installed.
I was having the same problem and fixed it.
I was running python 2.7.3 that I installed from Ninite.com.
Uninstalled python, downloaded and installed 2.7.5 installer directly from python.org and problem has gone away.