I am attempting to run a sqlite3 command from within Go, to do a simple query (I completly understand there are other ways to access sqlite from go, but thats not the issue here)
func sqliteTest(){
cmd:=exec.Command("/usr/bin/sqlite3", " /home/chadg/Downloads/testDb.db 'select * from testTable;'")
fmt.Println("Command String:",cmd.String())
out,err:=cmd.CombinedOutput()
if err!=nil{
fmt.Println("Error Accessing Database:",err.Error(),string(out))
return
}
fmt.Println("Result Length:",len(out))
fmt.Println("Result:",string(out))
}
Running this gives me this output:
Command String: /usr/bin/sqlite3 /home/chadg/Downloads/testDb.db 'select * from testTable;'
Result Length: 0
Result:
Process finished with exit code 0
However if I run the same command by hand:
$ /usr/bin/sqlite3 /home/chadg/Downloads/testDb.db 'select * from testTable;'
testName|testValue
Have certainly run into many issues when running commands from within go, but there is always an error generated, but this time, there is no error, but the result from stdout and stderr seems to both be empty.
EDIT:
Single quotes vs double quotes doesn't change anything
cmd:=exec.Command("/usr/bin/sqlite3", " /home/chadg/Downloads/testDb.db \"select * from testTable;\"")
still returns an empty output, but works as expected from the command line.
Arguments in exec.Command should be passed as separate strings.
cmd := exec.Command("/usr/bin/sqlite3", "/home/chadg/Downloads/testDb.db", "select * from testTable;")
As for the behavior you're observing:
When sqlite3 is started non-interactively with 1 argument, it treats that as the database file name and reads the query from stdin, but cmd.CombinedOutput() closes stdin, resulting in an empty output.
Related
I’m trying to make the move from SQL Server to Mariadb and I’m running into a lot of little quirks that drive me crazy at times. Chief among them are the apparent differences between executing a script from a file and pasting it into a console window. I was used to using tabs for formatting code in MSSQL, so I’m still trying to get used to the tab behavior in the Mariadb console, but at least that makes sense. The difference between var and #var is odd to me as well. The code below runs fine when execute through Pycharm, but fails when pasted into a console window. I’m running Mariadb (##version: 10.5.18-MariaDB-0+deb11u1) on a Pi Raspberry accessing it through SSH from a Windows 11 box.
All the script is doing is populating a table with random combinations of last names and male/female first names from U.S. census data for testing. Any help with the error would be greatly appreciated. I really want to understand why the difference between pasting and executing the file.
SET #loops := 10;
WHILE #loops > 0 DO
INSERT person(last_name, first_name, iso_country)
VALUES(census.random_name(0), census.random_name(FLOOR(1 + RAND() * (2 - 1 +1))), 'US');
SET #loops := #loops - 1;
END WHILE;
SELECT * FROM person;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'END WHILE' at line 1
I executed the file from the Pycharm IDE where it runs fine, and pasted the same code into the Mariadb console where the error was raised, and the insertions did not occur.
The mariadb command line client has it's own small parser, by default a semicolon is interpreted as end of statement.
With default delimiter ; your statement is splitted after first semicolon into WHILE #loops > 0 DO INSERT person(last_name, first_name, iso_country) VALUES(census.random_name(0), census.random_name(FLOOR(1 + RAND() * (2 - 1 +1))), 'US'); SET #loops := #loops - 1; and END WHILE. When sending both statements to the server, server will return 2 errors, but only last error is displayed.
Instead you should use a different delimiter:
DELIMITER $$
WHILE #loops > 0 DO
INSERT person(last_name, first_name, iso_country)
VALUES(census.random_name(0), census.random_name(FLOOR(1 + RAND() * (2 - 1 +1))), 'US');
SET #loops := #loops - 1;
END WHILE;$$
See also Delimiters
I have the following task in my DAG:
create_features_table = PostgresOperator(
task_id="create_features_table",
postgres_conn_id="featuredb",
sql="src/test.sql "
)
But when I test the task, I get this error:
psycopg2.errors.SyntaxError: syntax error at or near "src"
LINE 1: src/test.sql
The content of the test.sql script is:
CREATE TABLE test(
C1 int,
C2 int,
);
I can't point out the error in the syntax, but that's because it is my first DAG. Any help would be greatly appreciated.
If I run the script directly from the postgres container's psql using "\i src/text.sql" it works fine.
I have tested the connection from the airflow web server and the connection works.
I found that I had to put a space before closing the quotes to avoid a jinja2.exeptions.TemplateNotFound error, but haven't been able to find the syntax error.
According to documentation (https://airflow.apache.org/docs/apache-airflow-providers-postgres/stable/_api/airflow/providers/postgres/operators/postgres/index.html#airflow.providers.postgres.operators.postgres.PostgresOperator)
if you are defining your sql script path, it must ends with .sql
You have a white space in your path in the end so Operatort things it’s a query to be executed on your postgre, not file with query. You can see it in the response from postgre. Run this query on your postgre instance src/test.sql and you will get the same syntax error.
You can fix it easily by removing that white space
sql="src/test.sql"
classic case of getting bogged down in technical stuff when learning a new language. nothing to see here...
I'm trying to enable_load_extension in tcl and get "invalid command name "dbHandle"" as an error message. The traditional SELECT name FROM sqlite_master WHERE type='table'; query at the end works perfectly, I can't see what the problem is.
Perhaps compile options? the PRAGMA compile_options; call comes back with COMPILER=msvc-1900 ENABLE_FTS3 ENABLE_RTREE THREADSAFE=1 Given that I'm not seeing SQLITE_OMIT_LOAD_EXTENSION
(https://www.sqlite.org/compile.html#omit_load_extension) I should be ok?
sqlite3_db_config with SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION
looks useful, see https://www.sqlite.org/c3ref/db_config.html and https://www.sqlite.org/c3ref/c_dbconfig_enable_fkey.html but these don't seem to be part of the tcl interface.
package require sqlite3
sqlite3 dbhandle example.db
puts [dbhandle eval {PRAGMA compile_options;}]
# $dbHandle enable_load_extension 1 ; #can't read "dbHandle": no such variable
# dbHandle enable_load_extension 1 ; #invalid command name "dbHandle"
# dbHandle enable_load_extension true ; #invalid command name "dbHandle"
# next line is fine
puts [dbhandle eval {SELECT name FROM sqlite_master WHERE type='table';}]
I want to execute a batch file using People code in Application Engine Program. But The program have an issue returning Exec code as a non zero value (Value - 1).
Below is people code snippet below.
Global File &FileLog;
Global string &LogFileName, &Servername, &commandline;
Local string &Footer;
If &Servername = "PSNT" Then
&ScriptName = "D: && D:\psoft\PT854\appserv\prcs\RNBatchFile.bat";
End-If;
&commandline = &ScriptName;
/* Need to commit work or Exec will fail */
CommitWork();
&ExitCode = Exec("cmd.exe /c " | &commandline, %Exec_Synchronous + %FilePath_Absolute);
If &ExitCode <> 0 Then
MessageBox(0, "", 0, 0, ("Batch File Call Failed! Exit code returned by script was " | &ExitCode));
End-If;
Any help how to resolve this issue.
Best bet is to do a trace of the execution.
Thoughts:
Can you log on the the process scheduler you are running this on and execute the script OK?
Is the AE being scheduled or called at run-time?
You should not need to change directory as you are using a fully qualified path to the script.
you should not need to call "cmd /c" as this will create an additional shell for you application to run within, making debuging harder, etc.
Run a trace, and drop us the output. :) HTH
What about changing the working directory to D: inside of the script instead? You are invoking two commands and I'm wondering what the shell is returning to exec. I'm assuming you wrote your script to give the appropriate return code and that isn't the problem.
I couldn't tell from the question text, but are you looking for a negative result, such as -1? I think return codes are usually positive. 0 for success, some other positive number for failure. Negative numbers may be acceptable, but am wondering if Exec doesn't like negative numbers?
Perhaps the PeopleCode ChDir function still works as an alternative to two commands in one line? I haven't tried it for a LONG time.
Another alternative that gives you significant control over the process is to use java.lang.Runtime.exec from PeopleCode: http://jjmpsj.blogspot.com/2010/02/exec-processes-while-controlling-stdin.html.
Background: I am developing a rscript that pulls data from a mysql database, performs a logistic regression and then inserts the predictions back into the database. I want the entire system to be self contained in the script in case of database failure. This includes all mysql stored procedures that the script depends on to aggregate the data on the backend since these would be deleted in such a database failure.
Question: I'm having trouble creating a stored procedure from an R script. I am running the following:
mySQLDriver <- dbDriver("MySQL")
connect <- dbConnect(mySQLDriver, group = connection)
query <-
"
DROP PROCEDURE IF EXISTS Test.Tester;
DELIMITER //
CREATE PROCEDURE Test.Tester()
BEGIN
/***DO DATA AGGREGATION***/
END //
DELIMITER ;
"
sendQuery <- dbSendQuery(connect, query)
dbClearResult(dbListResults(connect)[[1]])
dbDisconnect(connect)
I however get the following error that seems to involve the DELIMITER change.
Error in .local(conn, statement, ...) :
could not run statement: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'DELIMITER //
CREATE PROCEDURE Test.Tester()
BEGIN
/***DO DATA AGGREGATION***/
EN' at line 2
What I've Done: I have spent quite a bit of time searching for the answer, but have come up with nothing. What am I missing?
Just wanted to follow up on this string of comments. Thank you for your thoughts on this issue. I have a couple Python scripts that need to have this functionality and I began researching the same topic for Python. I found this question that indicates the answer. The question states:
"The DELIMITER command is a MySQL shell client builtin, and it's recognized only by that program (and MySQL Query Browser). It's not necessary to use DELIMITER if you execute SQL statements directly through an API.
The purpose of DELIMITER is to help you avoid ambiguity about the termination of the CREATE FUNCTION statement, when the statement itself can contain semicolon characters. This is important in the shell client, where by default a semicolon terminates an SQL statement. You need to set the statement terminator to some other character in order to submit the body of a function (or trigger or procedure)."
Hence the following code will run in R:
mySQLDriver <- dbDriver("MySQL")
connect <- dbConnect(mySQLDriver, group = connection)
query <-
"
CREATE PROCEDURE Test.Tester()
BEGIN
/***DO DATA AGGREGATION***/
END
"
sendQuery <- dbSendQuery(connect, query)
dbClearResult(dbListResults(connect)[[1]])
dbDisconnect(connect)