If I try this query:
select * FROM sphinx.products where `query` = "test";
it works. But if I try to give it weights it returns an error:
select * FROM sphinx.products where `query` = "test;sort=extended:#weight DESC;weights=3,1,1,1";
Fails with error:
Error in query (1430): There was a problem processing the query on the foreign data source.
Data source error: searchd error: invalid deprecated unordered_weight count 4 (expe
(Error reported by MariaDB gets truncated there, but I believe it says "expecting 0")
And:
select * FROM sphinx.products where `query` = "test;sort=extended:#weight DESC";
Fails with error:
Error in query (1430): There was a problem processing the query on the foreign data source.
Data source error: searchd error: index 'sku_products': sort-by attribute '#weight'
(Again, error returned by SphinxSearch gets truncated by MariaDB)
All the documentation I find about SphinxSE tells me to query the index this way, yet it does not work, but nobody in the Internet seem to have met this error since nobody is asking about this anywhere...
Am I doing something wrong?
Well, the option weights= didn't work, but it accepted fieldweights=sku,90,partnumber,30,barcode,20,name,10.
(I.e., fieldweights=<field1_name>,<field1_weight>,...)
Results came ordered by weight even without specifying sort=extended:#weight DESC, so I dodged both errors and got what I needed.
Hope this helps anyone in the same situation.
Related
Say I have some SQL code that refreshes a table of data, and I would like to schedule an R script to schedule this code to run daily. Is there a way to capture any potential error messages the SQL code may throw and save that error message to an R variable instead of the error message being displayed in the R console log?
For an example, assume I have stored procedure sp_causing_error() in BigQuery that that takes data from a source table source_table and refreshes a target table table_to_refresh.
CREATE OR REPLACE PROCEDURE sp_causing_error()
BEGIN
CREATE OR REPLACE TABLE table_to_refresh AS (
Select non_existent_column, x, y, z
From source_table
);
END;
Assume the schema of the source_table has changed and column non_existent_column no longer exists. When attempting to call sp_causing_error() in RStudio via:
library(bigrquery)
query <- "CALL sp_causing_error()"
bq_project_query(my_project, query)
We get an error message printed to the console (which masks the actual error message we would encounter if running in BigQuery):
Error in UseMethod("as_bq_table") : no applicable method for 'as_bq_table' applied to an object of class "NULL"
If we were to run sp_causing_error() in BigQuery, it throws an error message stating:
Query error: Unrecognized name: non_existent_column at [sp_throw_error:3:8]
Are query error message displayed in BigQuery ever captured anywhere in bigrquery when executing SQL? My goal would be to have some sort of try/catch block in the R script that catches an error message that can then be written to an output file if the SQL code did not run successfully. Hoping there is a way we can capture the descriptive error message from BigQuery and assign it to an R variable for further processing.
UPDATE
R's tryCatch() function comes in handy here to catch the R error message:
query <- "CALL sp_causing_error()"
result <- tryCatch(
bq_project_query("research-01-217611", query),
error = function(err) {
return(err)
}
)
result now contains the error message from the R console:
<simpleError in UseMethod("as_bq_table"): no applicable method for 'as_bq_table' applied to an object of class "NULL">
However, this is still not descriptive of the actual error message we see if we execute the same SQL code in BigQuery, quoted above which references an unrecognized column name. Are we able to catch that error message instead of the more generic R error message?
UPDATE/ANSWER
Wrapping the stored procedure call within R using BigQuery's Begin...Exception...End syntax lets us get at the actual error message. Example code snippet:
query <- '
BEGIN
CALL sp_causing_error();
EXCEPTION WHEN ERROR THEN
Select 1 AS error_flag, ##error.message AS error_message, ##error.statement_text AS error_statement_text, ##error.formatted_stack_trace AS stack_trace
;
END;
'
query_result <- bq_table_download(bq_project_query(<project>, query))
error_flag <- query_result["error_flag"][[1]]
if (error_flag == 0) {
print("Job ran successfully")
} else {
print("Job failed")
# Access error message variables here and take additional action as desired
}
Warning: Note that this solution could cause an R error if the stored procedure completes successfully, as error_flag will not exist unless explicitly passed at the end of the stored procedure. This can be worked around by adding one line at the end of your stored procedure in BigQuery to set the flag appropriately so the bq_table_download() function will get a value upon the stored procedure running successfully:
BEGIN
-- BigQuery stored procedure code
-- ...
-- ...
Select 0 AS error_flag;
END;
I have to create an SQR that generates a list of EEIDs, if there were any changes to the Pension data in the past day. The SQR compiles and works perfectly when I hardcode in the table names.
However, when I tried using variables for the table names, I get a compile error
I've pasted the portion of SQR that I'm trying to fix
When I start using $tableName and $auditTableName as table variables, that's when I get the error and I'm not sure what is going wrong
Can anyone help?
Please and Thank You
!***************************
begin-procedure Process-Main
!***************************
let $tableName = 'PS_PENSION_PLAN'
let $auditTableName = 'PS_AUDIT_PENSION_PLN'
let $dummy-dyn-variable = ''
begin-SELECT DISTINCT
L.EMPLID
L.EMPL_RCD
do someProcName(&L.EMPLID, &L.EMPL_RCD)
FROM [$dummy-dyn-variable]
(
SELECT DISTINCT
PP.EMPLID,
PP.EMPL_RCD,
PP.EFFDT,
'1901-01-01 12:00:00' AS AUDIT_STAMP
FROM [$dummy-dyn-variable] [$tableName] PP
UNION
SELECT DISTINCT
A.EMPLID,
A.EMPL_RCD,
A.EFFDT,
A.AUDIT_STAMP
FROM [$dummy-dyn-variable] [$auditTableName] A
)L
WHERE DATEDIFF(DAY,CAST(L.AUDIT_STAMP AS DATE),SYSDATE) = 1
ORDER BY 1,2
end-SELECT
end-procedure
Edit:
does the UNION have anything to do with this?
I keep receiving is this error:
(SQR 5528) ODBC SQL dbdesc: SQLNumResultCols error 102 in cursor 1:
[Microsoft][ODBC SQL Server Driver][SQL Server]Incorrect syntax near 'FROM'.
(SQR 5528) ODBC SQL dbdesc: SQLNumResultCols error 8180 in cursor 1:
[Microsoft][ODBC SQL Server Driver][SQL Server]Statement(s) could not be prepared.
Edit2:
Ok, initial problem solved with [$dummy-dyn-variable], which led to the next problem with the DO command. I've updated the code above with DO someProcName(param_a, param_b)
I am now getting an error saying:
(SQR 2002) DO arguments do not match procedure's
Weird part, if I remove the dynamic table variables and hardcode the table names in the FROM section, then it compiles properly without errors. This makes me believe that the error is not related to my someProcName (maybe?)
am I missing something here?
I am not an admin so I can't change the scopes. I can send slackr_bot messages to a channel I set up in the creation of the app in UI but doing the below does not work. Has anyone found a solution to this?
I created a txt file called: test.txt
Within that txt file it looks like this:
api_token: xxxxxxxxxxxx
channel: #channel_name
username: myusername
incoming_webhook_url: https://hooks.slack.com/services/xxxxxxxxxxx/xxxxxxxxxxxxx
Then I want to simply send a message but eventually I would like to run the function
ggslackr(qplot(mpg, wt, data=mtcars))
slackr_setup(config_file = "test.txt")
my_message <- paste("I'm sending a Slack message at", Sys.time(), "from my R script.")
slackr_msg(my_message, channel = "#channel_name", as_user=F)
Here is the error message:
Error: Join columns must be present in data.
x Problem with `id`.
Run `rlang::last_error()` to see where the error occurred.
In addition: Warning message:
In structure(vars, groups = group_vars, class = c("dplyr_sel_vars", :
Calling 'structure(NULL, *)' is deprecated, as NULL cannot have attributes.
Consider 'structure(list(), *)' instead.
Edit #2:
Okay, I learned some things regarding packages. If I had to do this over, I'd have gone to their github repo and read the issue tracker.
The reason is that it appears that slackr has a few issues related to changes in Slack's API.
And also since there has been a large updating of R (version 4.x) a lot of packages got broken.
My sense is that our issue is with a line of code inside a slackr function (slackr_util.r--iirc) that calls a dplyr join that is looking for a particular id that does not exist.
So, I'm going to watch the issue tracker and see what comes of it.
Edit: Try slackr_bot(my_message,channel = "#general")
worked as advertised!
But ggslackr continues to fail.
I'm having the same issue. I've found in another thread a debugging start:
`rlang::last_error()`
When I run that,
Backtrace:
1. slackr::slackr_msg(my_message, channel = "#general")
5. slackr::slackr_chtrans(channel)
6. slackr::slackr_ims(api_token)
8. dplyr:::left_join.data.frame(users, ims, by = "id", copy = TRUE)
9. dplyr:::join_mutate(...)
10. dplyr:::join_cols(...)
11. dplyr:::standardise_join_by(by, x_names = x_names, y_names = y_names)
12. dplyr:::check_join_vars(by$y, y_names)
So, step 8 there is a join effort by id, which I suppose this implies that 'id' is missing.
yet, if I run from github issue tracker : slackr::slackrSetup(echo=TRUE) I get the following:
{
"SLACK_CHANNEL": ["#general"],
"SLACK_USERNAME": ["slackr_brian"],
"SLACK_ICON_EMOJI": ["NA"],
"SLACK_INCOMING_URL_PREFIX": ["https://hooks.xxxxxxx"],
"SLACK_API_TOKEN": ["token secret"]
}
I'm not sure where to go from here as the issue tracker conversation makes mention of confirming webhooks going to the correct channel and becomes very user specific.
So, that's as far as I have gotten.
I'm using R to plot some data I pull out of a database (the Stack Exchange data dump, to be specific):
dplyr::tbl(serverfault,
dbplyr::sql("
select year(p.CreationDate) year,
avg(p.AnswerCount*1.0) answers_per_question,
sum(iif(ClosedDate is null, 0.0, 100.0))/count(*) close_rate
from Posts p
where PostTypeId = 1
group by year(p.CreationDate)
order by year(p.CreationDate)
"))
The query works fine on SEDE, but I get this error in the R console:
Error: <SQL> 'SELECT *
FROM (
select year(p.CreationDate) year,
avg(p.AnswerCount*1.0) answers_per_question,
sum(iif(ClosedDate is null, 0.0, 100.0))/count(*) close_rate
from Posts p
where PostTypeId = 1
group by year(p.CreationDate)
order by year(p.CreationDate)
) "zzz11"
WHERE (0 = 1)'
nanodbc/nanodbc.cpp:1587: 42000: [FreeTDS][SQL Server]Statement(s) could not be prepared.
I reckoned "Statement(s) could not be prepared." meant that SQL Server didn't like the query for some reason. Unfortunately, it didn't give any hint about what went wrong. After fiddling with the query for a bit, I noticed it was wrapped in a subselect, according to the error message. Copying and executing the full query as constructed by one of the libraries in the chain, SQL Server gave me this more informative error message:
The ORDER BY clause is invalid in views, inline functions, derived tables, subqueries, and common table expressions, unless TOP, OFFSET or FOR XML is also specified.
Now the solution is obvious: remove (or comment out) the order by clause. But where is the detailed error message in the R console? I'm using Rstudio, should that matter. If I could get the full exception right next to the code I'm working on, it would help me fix bug a lot quicker. (And just to be clear, I get cryptic errors from dplyr::tbl often and typically use binary search debugging to fix them.)
I am using Teradata SQLA 14.01 and want to use the OTRANSLATE function, but it is not working. I have checked the DBC.FunctionsV table to ensure that the function is defined in the system, which it is. However, even when I use the sample code;
SELECT OTRANSLATE('TD13.0 is the current database version','3', '4');
I am faced with a 3706 error:
Syntax error: expected something between '(' and the string 'T'
keyword.
All help will be greatly appreciated!!