Perl dancer error execute method undefined - sqlite

I was trying to make a website using perl dancer, below is my code. It seems to be correct but the page keeps loading and never enters the values in the database. When I cancel the page I get an error stating "request to POST /appform crashed: Can't call method "execute" on an undefined value". I can't figured out whats wrong in the code. If you have any other code please mention.
I am using SQLite for database.
There is a database campus.dband I am inserting the value in student table.
post '/appform' => sub {
my $q = CGI ->new;
my $name = $q->param ("firstname");
my $password = $q->param("password");
my $mobile_no = $q->param("mobile");
my $gender = $q->param("gender");
my $email = $q->param("email");
my $address = $q->param("address");
my $sslc = $q->param("SSLC");
my $hsc = $q->param("HSC");
my $cgpa = $q->param("cgpa");
my $languages = $q->param("lang");
my $internships = $q->param("intern");
my $preferred_loc = $q->param("country");
my $sql = "insert into student(name,mobile_no,gender,email,address,sslc,hsc,cgpa,languages,internships,preferred_loc,password,applied_job,company_applied) values ('?','?','?','?','?','?','?','?','?','?','?','?','?','?');";
my $sth = database->prepare($sql);
$sth->execute($name,$mobile_no,$gender,$email,$address,$sslc,$hsc,$cgpa,$languages,$internships,$preferred_loc,$password) or die $sth->errstr;
#$sth->execute();
$sth-> finish;
set_flash('New entry posted!');
redirect '/';
};

You're using the database keyword to get a database handle. I'm guessing that's coming from Dancer2::Plugin::Database (it would be useful if you could include information like this in your question).
The error says that you're calling execute() on an undefined value. You're calling execute() on the variable $sth. So $sth is undefined. You get $sth by calling prepare() on the database handle returned from database(). So it looks like the prepare() call is failing. You should check the return value from that call and throw an error if it fails.
The most common reason for prepare() to fail is that you're trying to compile an SQL statement that contains an error. I can't see any obvious error in your SQL, but it's worth checking it by running it manually against your database.
I see you're using bind params in your SQL statement. That's a great idea, but please note that you don't need to quote the question marks in your SQL - the database driver will handle that for you. I don't think that's what is causing your problem though.
I also see that you're using CGI.pm inside your Dancer app to get the request parameters. To be honest, I'm slightly surprised that it works - but it's a terrible idea. Dancer has its own keywords that will give you this information. Look at query_parameters(), body_parameters() and route_parameters() in the Dancer documentation.

In addition to the points made already, that your DBI prepare() call is probably failing (add error-checking to see why, e.g. my $sth = database->prepare('...') or die "DB error: " . database->errstr) and that you're using CGI.pm within a Dancer app (... don't do that, I'm surprised it would work at all - look at the Dancer documentation for how to access the params your app was sent), look also at the quick_insert convenience method provided by Dancer::Plugin::Database / Dancer2::Plugin::Database so that you don't have to write that SQL INSERT statement at all.

Related

Doctrine setParameter always produces "?"

I would like to compare a datetime value (from a database) with the current time and check whether the current time is before or after the value in the database. This is done in a Symfony project. I tried to follow the instructions on the Symfony website.
So I wrote the following Doctrine query in a Repository Class which checks for a user lockout time and checks if it still lies in the future:
$user_id = 1; // Just giving $user_id a value for this example
$qb = $this->createQueryBuilder('user')
->andWhere('user.lockout_time > :time')
->setParameter('time', date("Y-m-d H:i:s"))
->andWhere('user.user_id = :user_id')
->setParameter('user_id', $user_id);
$query = $qb->getQuery();
echo $query->getSQL();
die;
When running this, both Where clauses contain "?" in the comparative value (e.g. WHERE user.lockout_time > ?). Obviously I want the actual values to be used in the query.
Initially I thought the date() function might be the issue, but even if I just use the :user_id I get the above error.
If I write ->andWhere('user.user_id = 1') I get the desired result.
If I replace :time with some date in the format 'Y-m-d H:i:s', I get the message "Error: Expected end of string..." (with the ... being the value for the Hour).
So both my setParameter() lines are not passing the values set in them. What am I overlooking?
Edit:
The suggested question here is not a duplicate. That just helps me see the query that is sent off. It was helpful in preparation of this question.
So here my own answer after some (= way too much) time of digging around.
The "?" are escaped values and do not represent what is actually in the query (which is another reason the above link doesn't help). To resolve this I resorted to monitoring the MySQL general log.
Here how to get to it, if someone has the same question. This log shows the actual SQL query.

How can I log sql execution results in airflow?

I use airflow python operators to execute sql queries against a redshift/postgres database. In order to debug, I'd like the DAG to return the results of the sql execution, similar to what you would see if executing locally in a console:
I'm using psycop2 to create a connection/cursor and execute the sql. Having this logged would be extremely helpful to confirm the parsed parameterized sql, and confirm that data was actually inserted (I have painfully experiences issues where differences in environments caused unexpected behavior)
I do not have deep knowledge of airflow or the low level workings of the python DBAPI, but the pscyopg2 documentation does seem to refer to some methods and connection configurations that may allow this.
I find it very perplexing that this is difficult to do, as I'd imagine it would be a primary use case of running ETLs on this platform. I've heard suggestions to simply create additional tasks that query the table before and after, but this seems clunky and ineffective.
Could anyone please explain how this may be possible, and if not, explain why? Alternate methods of achieving similar results welcome. Thanks!
So far I have tried the connection.status_message() method, but it only seems to return the first line of the sql and not the results. I have also attempted to create a logging cursor, which produces the sql, but not the console results
import logging
import psycopg2 as pg
from psycopg2.extras import LoggingConnection
conn = pg.connect(
connection_factory=LoggingConnection,
...
)
conn.autocommit = True
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)
logger.addHandler(logging.StreamHandler(sys.stdout))
conn.initialize(logger)
cur = conn.cursor()
sql = """
INSERT INTO mytable (
SELECT *
FROM other_table
);
"""
cur.execute(sql)
I'd like the logger to return something like:
sql> INSERT INTO mytable (
SELECT ...
[2019-07-25 23:00:54] 912 rows affected in 4 s 442 ms
Let's assume you are writing an operator that uses postgres hook to do something in sql.
Anything printed inside an operator is logged.
So, if you want to log the statement, just print the statement in your operator.
print(sql)
If you want to log the result, fetch the result and print the result.
E.g.
result = cur.fetchall()
for row in result:
print(row)
Alternatively you can use self.log.info in place of print, where self refers to the operator instance.
Ok, so after some trial and error I've found a method that works for my setup and objective. To recap, my goal is to run ETL's via python scripts, orchestrated in Airflow. Referring to the documentation for statusmessage:
Read-only attribute containing the message returned by the last command:
The key is to manage logging in context with transactions executed on the server. In order for me to do this, I had to specifically set con.autocommit = False, and wrap SQL blocks with BEGIN TRANSACTION; and END TRANSACTION;. If you insert cur.statusmessage directly following a statement that deletes or inserts, you will get a response such as 'INSERT 0 92380'.
This still isn't as verbose as I would prefer, but it is a much better than nothing, and is very useful for troubleshooting ETL issues within Airflow logs.
Side notes:
- When autocommit is set to False, you must explicitly commit transactions.
- It may not be necessary to state transaction begin/end in your SQL. It may depend on your DB version.
con = psy.connect(...)
con.autocommit = False
cur = con.cursor()
try:
cur.execute([some_sql])
logging.info(f"Cursor statusmessage: {cur.statusmessage})
except:
con.rollback()
finally:
con.close()
There is some buried functionality within psycopg2 that I'm sure can be utilized, but the documentation is pretty thin and there are no clear examples. If anyone has suggestions on how to utilize things such as logobjects, or returning join PID to somehow retrieve additional information.

Using sqlite database with qt

Here is my code, there doesn't seem to be anything wrong:
QSqlDatabase db=QSqlDatabase::addDatabase("QSQLITE");
db.setDatabaseName("thedata.sqlite");
db.open();
QSqlQuery query;
query.prepare("SELECT lastname FROM people where firstname='?' ");
query.bindValue(0, lineEdit->text());
bool x = query.exec();
if(x){
lineEdit_2->setText(query.value(0).toString());
}
else {
QSqlError err;
err = query.lastError();
QMessageBox::about(this,"error",err.text() );
}
When the program is working always it gives the error parameter count mismatch I'm using qt 4.8 and its own headers for using sqlite.
I would be very thankful for any advice, though I searched in google i see many posts in this issue but nothing helped me.
Thank you.
You're prepared statement is wrong, it should be:
quary.prepare("SELECT lastname FROM people where firstname=?");
Notice that there are no single quotes (') around the placeholder. If you put the quotes, it gets passed as a literal to the database, leaving you with a parameter-less query and code that passes too many parameters.
(Changing that variable name to query would be a nice touch too.)
Also you need to check the return value if QSqlQuery::prepare, and print out/display the error message you're getting from that if it fails – otherwise QSqlQuery::exec resets the current error and you'll get a pretty meaningless error message if there was a problem parsing the prepared statement.
if(x){
lineEdit_2->setText(quary.value(0).toString());
}
This is incorrect too. The you need to call (and check the return value of) query.next() to position the result set to the first row returned (if there is one). You can't use .value(X) before you've called .next().

Sqlite pending operation occuring while fetching data from database

My code is:
Statement mstatement = null;
URI uri = URI.create(DBPath);
sqliteDb = DatabaseFactory.openOrCreate(uri);
mStatement = sqliteDb.createStatement(strQuery);
mStatement.prepare();
My problem is, I am getting error as "Sqlite pendingOperation" at mstatement.prepare();. Please tell me why I am getting this and how to remove it. One more thing, I am using this code in multiple threads.

Does "insert" in SQLite return SQLITE_OK or SQLITE_DONE?

What does statement "insert" in SQLite return in case of success?
I always believed that it should be SQLITE_DONE, but recently in my logs I found out the following string:
sqlite3_step error: 'not an error'
And here is the code that logs the mentioned string:
prepareStatement(addTranslationStmt2, "INSERT INTO translations(lang1_wordid, lang2_wordid) VALUES(?, ?)");
if (!addTranslationStmt2) return -2;
sqlite3_bind_int(addTranslationStmt2, 1, word_id);
sqlite3_bind_int(addTranslationStmt2, 2, translation_id);
if(sqlite3_step(addTranslationStmt2) != SQLITE_DONE)
{
NSLog(#"sqlite3_step error: '%s'", sqlite3_errmsg(database));
sqlite3_reset(addTranslationStmt2);
return -1;
}
sqlite3_reset(addTranslationStmt2);
I am wondering, why does it work in most cases.
Should I change SQLITE_DONE in my code to SQLITE_OK?
Thanks.
SQLITE_DONE
http://www.sqlite.org/c3ref/step.html
You could also try printing out the error code to find out what the problem is.
In cases like these, I like to look at code samples. Here are some good ones:
http://sqlite.phxsoftware.com/forums/p/76/6659.aspx
The SQLite Result Codes Reference lists SQLITE_OK as indicating a successful result. It is also the first error code, having an error code of 0, making it the canonical result (i.e. the result I would expect on a successful operation).
You should put a breakpoint or print statement in your code to find out if it really is returning zero, and check your data to make sure you're getting the result you expect. If that all checks out, I would change your condition to check for SQLITE_OK.
The details of the behavior of the sqlite3_step() interface depend on whether the statement was prepared using the newer "v2" interface sqlite3_prepare_v2() and sqlite3_prepare16_v2() or the older legacy interface sqlite3_prepare() and sqlite3_prepare16().
In the legacy interface, the return value will be either SQLITE_BUSY, SQLITE_DONE, SQLITE_ROW, SQLITE_ERROR, or SQLITE_MISUSE. With the "v2" interface, any of the other result codes or extended result codes might be returned as well.
Switching from "Debug" configuration to "Release" resolved the issue for me.

Resources