When creating a tag with GEOPOINT, the query cannot be performed in the nGQL, and it reported: Storage Error: Edge prop not found.
E0712 13:16:26.017154 34888 RowReaderV2.cpp:190] Geography::fromWKB failed: Failed to parse an valid Geography instance from the wkb `'
E0712 13:16:26.017154 34888 RowReaderV2.cpp:190] Geography::fromWKB failed: Failed to parse an valid Geography instance from the wkb `'
E0712 13:16:26.471570 34888 RowReaderV2.cpp:190] Geography::fromWKB failed: Failed to parse an valid Geography instance from the wkb `'
E0712 13:16:26.773860 34888 RowReaderV2.cpp:190] Geography::fromWKB failed: Failed to parse an valid Geography instance from the wkb `'
E0712 13:16:27.073959 34888 RowReaderV2.cpp:190] Geography::fromWKB failed: Failed to parse an valid Geography instance from the wkb `'
Statement to create a node;
INSERT VERTEX IF NOT EXISTS CONCEPT (id, name, iri, nicks, source_type, source_name, source_batch, create_user, update_user, create_time, update_time) VALUES
"75c5eebd3eaea875c822ad5598aef065":("75c5eebd3eaea875c822ad5598aef065", "AAA", "https://g.w/test#AAA", NULL, "MANUAL", "test", "test-0711", "jal", "jal", datetime("2022-03-22 11:10:00.000"), datetime("2022-07-01 20:29:54.342")),
"91b32acd631cfdcc171ce93":("91b32acd631cfdcc171ce93", "BBB", "https://g.w/test#BBB", NULL, "MANUAL", "test", "test-0711", "jal", "jal", datetime("2022-03-22 11:10:00.000"), datetime("2022-07-01 20:29:54.342"))
Statement to create an edge:
INSERT EDGE RELATETO (id, ontrelid, ontreliri, name, nicks, weight, time_start, time_finish, address_point, address_name, address_detail, source_type, source_name, source_batch, create_user, update_user, create_time, update_time) VALUES "75c5eebd3eaea875c822ad5598aef065"->"cb3a063a491b32acd631cfdcc171ce93"#0:("d6ba99a8aafcd2a6e6bc6e4c46745c82", "74004a7a62f3dd1fe160f02203c6933c", null, "居住", "", 0.0, null, null, ST_Point(39.657589, 116.070870), "房山区", "世界名园A区", "MANUAL", "el51:9101", "20220623", "el51:9101", "el51:9101", datetime("2022-03-22 11:10:00.000"), datetime("2022-07-01 20:29:54.342"));
Query statement (both OpenCypher and nGQL were tried).
MATCH p=(m:ENTITY {id: "5f136053f62099a86cb15d2121f2dace"})-[r:RELATETO]-(n:ENTITY) RETURN p
MATCH p=(v)-[e:RELATETO*1]-(v2)
WHERE id(v) IN ["75c5eebd3eaea875c822ad5598aef065"]
RETURN p LIMIT 100
Related
Here is my function
getSQL <- function(server="server name", database="database name", Uid="
user name", Pwd="password", Query){
conlink <- paste('driver={SQL Server};server=', server,';database=',database,';Uid=', Uid,
';Pwd=', Pwd,';Encrypt=True;TrustServerCertificate=False', sep="")
conn <- odbcDriverConnect(conlink)
dat <- sqlQuery(channel= conn, Query, stringsAsFactors = F)
odbcCloseAll()
return(dat)
}
When I call the function using
query.cut = "SELECT [measurename]
,[OrgType]
,[year_session]
,[Star]
,[cutvalue]
,[Date]
,[File]
FROM [database name].[dbo].[DST_Merged_Cutpoint]
ORDER BY [year_session] DESC
"
getSQL(Query=query.cut)
I get this error:
Error in sqlQuery(conn, Query, stringsAsFactors = F) :
first argument is not an open RODBC channel
In addition: Warning messages:
1: In odbcDriverConnect(conlink) :
[RODBC] ERROR: state 28000, code 18456, message [Microsoft][ODBC SQL Server Driver][SQL Server]Login failed for user ' insightm8'.
2: In odbcDriverConnect(conlink) :
[RODBC] ERROR: state 01S00, code 0, message [Microsoft][ODBC SQL Server Driver]Invalid connection string attribute
3: In odbcDriverConnect(conlink) :
Error in sqlQuery(conn, Query, stringsAsFactors = F) :
first argument is not an open RODBC channel
How can I fix these errors? Thanks in advance
Take care not to add spaces to UID:
Server]Login failed for user ' insightm8'.
Reproducing this on an SQL Server connection creates the same error.
Try using paste0 instead of paste :
conlink <- paste0('driver={SQL Server};server=', server,';database=',database,';Uid=', Uid,
';Pwd=', Pwd,';Encrypt=True;TrustServerCertificate=False', sep="")
Airflow-pandas-read-sql-query to dataframe
i am trying to connect to SQL server local to get data from a table and process the data using pandas operations but i m failing to figure out how to pass the select query results to a data frame
the below works to clear data in the table
``` sql_command = """ DELETE FROM [TestDB].[dbo].[PythonTestData] """
t3 = MsSqlOperator( task_id = 'run_test_proc',
mssql_conn_id = 'mssql_local',
sql = sql_command,
dag = dag,
database = 'TestDB',
autocommit = True) ```
the intended pandas is
query = 'SELECT * FROM [ClientData] '#where product_name='''+i+''''''
df = pd.read_sql(query, conn)
pn_list = df['ClientID'].tolist()
#print("The original pn_list is : " + str(pn_list))
for i in pn_list:
varw= str(i)
queryw = 'SELECT * FROM [ClientData] where [ClientID]='''+varw+''
dfw = pd.read_sql(queryw, conn)
dfw = dfw.applymap(str)
cols=['product_id','product_name','brand_id']
x=dfw.values.tolist()
x=x[0]
ClientID=x[0]
Name=x[1]
Org=x[2]
Email=x[3]
#print('Name :'+Name+' ,'+'Org :'+Org+' ,'+'Email :'+Email+' ,'+'ClientID :'+ClientID)
salesData_qry= 'SELECT * FROM [TestDB].[dbo].[SalesData] where [ClientID]='''+ClientID+''
salesData_df= pd.read_sql(salesData_qry, conn)
salesData_df['year1'] = salesData_df['Order Date'].dt.strftime('%Y')
salesData_df['OrderMonth'] = salesData_df['Order Date'].dt.strftime('%b')
filename ='Daily_Campaign_Report_'+Name+'_'+Org+'_'+datetime.now().strftime("%Y%m%d_%H%M%S")
p = Path('C:/Users/user/Documents/WorkingData/')
salesData_df.to_csv(Path(p, filename + '.csv'))```
Please point me to correct approach as i m new to airflow
I'm not so clear on how you generate the query code but in order to get dataframe from MsSQL you need to use MsSqlHook:
from airflow.providers.microsoft.mssql.hooks.mssql import MsSqlHook
def mssql_func(**kwargs):
hook = MsSqlHook(conn_id='mssql_local')
df = hook.get_pandas_df(sql="YOUR_QUERY")
#do whatever you need on the df
run_this = PythonOperator(
task_id='mssql_task',
python_callable=mssql_func,
dag=dag
)
this is the code i am using for the dag
def mssql_func(**kwargs):
conn = MsSqlHook.get_connection(conn_id="mssql_local")
hook = conn.get_hook()
df = hook.get_pandas_df(sql="SELECT * FROM [TestDB].[dbo].[ClientData]")
#do whatever you need on the df
print(df)
run_this = PythonOperator(
task_id='mssql_task',
python_callable=mssql_func,
dag=dag
)
Error Log
[2021-01-12 16:07:15,114] {providers_manager.py:159} WARNING - The provider for package 'apache-airflow-providers-imap' could not be registered from because providers for that package name have already been registered
[2021-01-12 16:07:15,618] {base.py:65} INFO - Using connection to: id: mssql_local. Host: localhost, Port: 1433, Schema: dbo, Login: sa, Password: XXXXXXXX, extra: None
[2021-01-12 16:07:15,626] {taskinstance.py:1396} ERROR - (18456, b"Login failed for user 'sa'.DB-Lib error message 20018, severity 14:\nGeneral SQL Server error: Check messages from the SQL Server\nDB-Lib error message 20002, severity 9:\nAdaptive Server connection failed (localhost)\nDB-Lib error message 20002, severity 9:\nAdaptive Server connection failed (localhost)\n")
Traceback (most recent call last):
File "src/pymssql.pyx", line 636, in pymssql.connect
File "src/_mssql.pyx", line 1964, in _mssql.connect
File "src/_mssql.pyx", line 682, in _mssql.MSSQLConnection.__init__
File "src/_mssql.pyx", line 1690, in _mssql.maybe_raise_MSSQLDatabaseException
_mssql.MSSQLDatabaseException: (18456, b"Login failed for user 'sa'.DB-Lib error message 20018, severity 14:\nGeneral SQL Server error: Check messages from the SQL Server\nDB-Lib error message 20002, severity 9:\nAdaptive Server connection failed (localhost)\nDB-Lib error message 20002, severity 9:\nAdaptive Server connection failed (localhost)\n")
I want to access the GetDeepSearchResults info from the Zillow API.
My code:
library(ZillowR)
zapi_key = getOption('Myapikey')
GetDeepSearchResults(
address = '600 S. Quail Ct.',
zipcode = '67114',
rentzestimate = FALSE,
api_key = zapi_key
)
Error:
Error in GetDeepSearchResults(address = "600 S. Quail Ct.", zipcode = "67114", :
unused arguments (zipcode = "67114", api_key = zapi_key)
Why does this error occur? What can I do to fix this?
Edit: I changed the code according to the comments and got this:
My code:
library(ZillowR)
zapi_key = getOption('myapikey')
GetDeepSearchResults(
address = '600 S. Quail Ct.',
citystatezip = '67114',
rentzestimate = FALSE,
zws_id = 'myapikey',
url = "http://www.zillow.com/webservice/GetDeepSearchResults.htm"
)
Output:
$request
$request$address
NULL
$request$citystatezip
NULL
$message
$message$text
[1] "Error: invalid or missing ZWSID parameter"
$message$code
[1] "2"
$response
NULL
How can I fix this?
The unused arguments error is typical when you pass arguments, which are not parts of the function. So R doesn't know what to do with those and returns the error. You can check the documentation of the function with ?GetDeepSearchResults
This shows you the usage:
GetDeepSearchResults(address = NULL, citystatezip = NULL,
rentzestimate = FALSE, zws_id = getOption("ZillowR-zws_id"),
url = "http://www.zillow.com/webservice/GetDeepSearchResults.htm")
To have this work, you have to set your id first with (you can create an id on https://www.zillow.com/howto/api/APIOverview.htm):
set_zillow_web_service_id("youractualkey")
So you function does not have the argument zipcode and api_key. Let's change your arguments to some which exist:
GetDeepSearchResults(address='600 S. Quail Ct.', citystatezip ='67114',
rentzestimate=FALSE)
You surely recognized I did not use your api_key. This is because the default:zws_id = getOption("ZillowR-zws_id") calls your global 'ZillowR-zws_id' which you just set with the set_zillow_web_service_id() command. So it's not necessary to change the default value. But you can skip this when you use zws_id ="youractualkey" from zillow
I made a random account I set up for validating. This gives me the output:
$request
$request$address
NULL
$request$citystatezip
NULL
$message
$message$text
[1] "Error: this account is not authorized to execute this API call"
$message$code
[1] "6"
$response
NULL
So I could successfully contact the server and my key was recognized. The account authority is not R related and has to be set on the website.
I've created a model in R, published into SQL Server table and validated the model by calling it into SQL Server.
However, I'm failing in an attempt to use the model for prediction over new data.
Here's my script:
DROP PROCEDURE IF EXISTS predict_risk_new_data;
GO
CREATE OR ALTER PROCEDURE predict_risk_new_data (#q nvarchar(max))
AS
BEGIN
DECLARE #model varchar(30) = 'risk_rxLogit'
DECLARE #rx_model varbinary(max) = (SELECT model FROM rx_models WHERE model_name = #model);
EXEC sp_execute_external_script
#language = N'R'
,#script = N'
require("RevoScaleR");
input_data = InputDataSet;
model <- unserialize(rx_model);
prediction <- rxPredict(model, input_data, writeModelVars = TRUE);
OutputDataSet <- cbind(predictions[1], predictions[2]);'
,#input_data_1 = #q
,#parallel = 1
,#params = N'#rx_model varbinary(max), #r_rowsPerRead int'
,#input_data_1_name = N'InputDataSet'
,#rx_model = #rx_model
,#r_rowsPerRead = 100
WITH result sets (("Risk_Pred" float, "ZIP" int));
END
GO;
/*
EXEC predict_risk 'SELECT TOP 100 [ZIP], [Week], [Age], [Risk] FROM dbo.Risk'
*/
Here's the error output:
Msg 39004, Level 16, State 20, Line 223 A 'R' script error occurred
during execution of 'sp_execute_external_script' with HRESULT
0x80004004. Msg 39019, Level 16, State 2, Line 223 An external script
error occurred: Error in unserialize(rx_model) : read error Calls:
source -> withVisible -> eval -> eval -> unserialize
Error in execution. Check the output for more information. Error in
eval(expr, envir, enclos) : Error in execution. Check the output
for more information. Calls: source -> withVisible -> eval -> eval ->
.Call Execution halted
New to R/ML in SQL Server, help would be aprreciated.
Thanks in advance.
When I did something like this I had to add as.raw to the model.
Try this
model <- unserialize(as.raw(rx_model));
I try to download the "World Gross domestic product, current prices" using the IMFdata-API. If I use this query:
library(IMFData)
databaseID <- 'IFS'
startdate = '2001-01-01'
enddate = '2016-12-31'
checkquery = TRUE
queryfilter <- list(CL_FREA="", CL_AREA_IFS="W0", CL_INDICATOR_IFS="NGDP_USD")
W0.NGDP.query <- CompactDataMethod(databaseID, queryfilter, startdate, enddate, checkquery)
I get this error:
Error: lexical error: invalid char in json text.
<?xml version="1.0" encoding="u
(right here) ------^
In addition: Warning message:
JSON string contains (illegal) UTF8 byte-order-mark!
How can I fix this? Is there a better way using Quandl, etc.?