SQLite CGI Programming (SQLITE_CANTOPEN) - sqlite

When I am trying to use sqlite3_exec to insert new data into the database, it returns error 14 (SQLITE_CANTOPEN). But when I am using sqlite3_prepare_v2 to select, it works fine. Is there an issue with permissions? How to fix it?
sprintf(temp, "INSERT INTO owned (pid, oname, okey, ohp, oatt, odef) VALUES (%d, %c%s%c, %c%s%c, %d, %d, %d);", pid, 34, poname, 34, 34, passkey, 34, sqlite3_column_int(res3,0), sqlite3_column_int(res3,1), sqlite3_column_int(res3,2));
error = sqlite3_exec(conn, temp, 0, 0, 0);

Related

Airflow failed to get task instance

i have tried to run a simple task using airflow bash operator but keep getting stuck on my DAG never stop running, it stays like green forever without success or fail, when i check the logs i see something like this. Thanks in advance for your time and answers
**`your text`**airflow-scheduler_1 | [SQL: INSERT INTO task_fail (task_id, dag_id, execution_date, start_date, end_date, duration) VALUES (%(task_id)s, %(dag_id)s, %(execution_date)s, %(start_date)s, %(end_date)s, %(duration)s) RETURNING task_fail.id]
airflow-scheduler_1 | [parameters: {'task_id': 'first_task', 'dag_id': 'LocalInjestionDag', 'execution_date': datetime.datetime(2023, 1, 20, 8, 0, tzinfo=Timezone('UTC')), 'start_date': datetime.datetime(2023, 1, 23, 3, 35, 27, 332954, tzinfo=Timezone('UTC')), 'end_date': datetime.datetime(2023, 1, 23, 3, 35, 27, 710572, tzinfo=Timezone('UTC')), 'duration': 0}]
postgres_1 | 2023-01-23 03:55:59.712 UTC [4336] ERROR: column "execution_date" of relation "task_fail" does not exist at character 41"""
I have tried with execution_datetime , using xcom_push and creating functions with xcom and changing to python operator but everything still fall back to same error

Are my unacknowledged kafka messages dropped due to network latency?

I have rdkafka producer written the following way, where localhost:9095 is tunneled to a digital ocean server in Germany with a kafka broker, whereas i, the localhost, am in New York.
I am noticing that at around ~150 miliseconds of thread::sleep(Duration::from_millis(150)) the last few of my messages stop being acknowledged by the callback (messages 8 and 9 for example),as if they didn't arrive. I know they did because i see the Kafka topic.
I'm wondering if my guess is correct that it is solely the latency between NY and Germany that dictates how long the broker takes to respond or if there is some config max.response.wait.ms setting i can set for acknowledgements to arrive sooner.
Of course if i increase the sleep time to ~200ms and higher i can see each message logged loud and clear. If i decrease sleep to 50ms the producer drops before any messages are sent and they never arrive to the topic... right?
let producer :ThreadedProducer< ProducerLogger> = ClientConfig::new()
.set("bootstrap.servers", "localhost:9095")
.create_with_context(ProducerLogger{})
.expect("Failed to create producer");
for i in 1..10{
let mut data = [0u8; 8];
rand::thread_rng().fill_bytes(&mut data);
println!("Sedndg msg with data {:x?}", data);
producer.send(
BaseRecord::to("rt-test")
.key(&format!("key-{}",i))
.payload(&format!("payload-{}",i))
)
.expect("couldn't send message");
// producer.flush(Duration::from_secs(1));
thread::sleep(Duration::from_millis(150)) // <--- Parameter to play with
}
Sedndg msg with data [da, 44, 1c, f0, 6, 1d, da, 9b]
Sedndg msg with data [37, 82, b2, 58, e2, 91, 40, 33]
Sedndg msg with data [8d, f8, 5c, df, 68, 12, a1, 26]
Sedndg msg with data [bd, 13, 7c, 81, 62, 5c, 93, 75]
Produced message key-1 successfully. Offset: 193. Partition :0
Produced message key-2 successfully. Offset: 194. Partition :0
Produced message key-3 successfully. Offset: 195. Partition :0
Produced message key-4 successfully. Offset: 196. Partition :0
Sedndg msg with data [d, 9d, 1c, 73, a7, 9f, b4, 2]
Produced message key-5 successfully. Offset: 197. Partition :0
Sedndg msg with data [c9, 1a, 3b, 8c, 31, cc, 84, f4]
Produced message key-6 successfully. Offset: 198. Partition :0
Sedndg msg with data [8a, 33, 26, 92, 2a, bf, d1, 7d]
Produced message key-7 successfully. Offset: 199. Partition :0
Sedndg msg with data [78, eb, e2, 41, 8d, b9, 29, 68]

Finstr get_xbrl_statement error parsing Extracted XBRL Instance Document

Most recent AAPL 10-k XBRL Instance Document for example:
doc <- "https://www.sec.gov/Archives/edgar/data/320193/000032019319000119/0000320193-19-000119-index.htm"
Run xbrlDoAll and xbrl_get_statements from XBRL and finstr packages, respectively
get_xbrl_doc <- xbrlDoAll(doc)
statements <- xbrl_get_statements(get_xbrl_doc)
Error: Each row of output must be identified by a unique combination of keys.
Keys are shared for 34 rows:
* 6, 8
* 5, 7, 9
* 49, 51
* 48, 50
* 55, 57
* 54, 56
* 11, 13
* 10, 12
* 25, 27
* 24, 26
* 59, 61
* 58, 60
* 29, 31
* 28, 30
* 63, 64, 66
* 62, 65
This sequence works perfectly up until 2019 when Apple switched to "Extracted XBRL Instance Document" from "XBRL Instance Document". Has anybody found a work around?
Not only have they switched that, but now the fact ID brings some kind of 32-digit key that seems to have no connections with any of the other files in the taxonomy. This is probably related to the iXBRL process and I've seen it with another company as well.
However, if the problem is just the word "Extracted", then you just need to change the script, but I'm guessing this is not the case and the solution relates to finding out what those 32-digit keys mean.
I'm not an R user and I don't use Finstr, but I guess we have the same problem. So, the answer to your question would be: you need to write your own parser now or wait until someone finishes theirs.

Debian not taking daylight saving time into account in R

I use following code to find whether daylight saving is in use in Central Europe in day given by variables year, month and day.
timeString = paste(toString(year), formatC(month, width = 2, flag="0"), formatC(day, width = 2, flag="0"), "12", sep = "-")
time = strptime(timeString, format = "%Y-%m-%d-%H")
diff = as.numeric(as.POSIXct(time, tz="UTC") - as.POSIXct(time, tz="Europe/Prague"))
On my PC (Ubuntu 16.04), diff is 2 when daylight saving is active, 1 othervise, but on server with Debian 8.8 it is 1 in all cases. Do you know how to set-up the server to behave as Ubuntu? Thanks.
Update: The change of Debian time settings would also change time used for crontab, which is undesirable. Reinstaling R with new configuration seemed risky becase there runs a few R script operationally every few minutes. So i chose "ugly" solution in form of R function:
DaylightSaving = function(year, month, day) {
# years 2010-2030
if (year < 2010 || year > 2030) {
stop("The function is implemented now only for years 2010-2030")
}
dayStart = c(28, 27, 25, 31, 30, 29, 27, 26, 25, 31, 29, 28, 27, 26,
31, 30, 29, 28, 26, 25, 31)
dayEnd = c(31, 30, 28, 27, 26, 25, 30, 29, 28, 27, 25, 31, 30, 29,
27, 26, 25, 31, 29, 28, 27)
if (month < 3 || month > 10) {
return(FALSE)
} else if (month == 3 && day < dayStart[year - 2009]) {
return(FALSE)
} else if (month == 10 && day >= dayEnd[year - 2009]) {
return(FALSE)
}
return(TRUE)
}
First of all, if you want to check whether daylight saving is in use, you can simply do:
#Make a test date
atime <- as.POSIXct("2017-05-23 13:25",
tz = "Europe/Prague")
#test for DST
as.POSIXlt(atime)$isdst > 0
The POSIXlt class is internally a list with an element isdst that is 0 if daylight saving time is not active, positive when it is, and negative when that information is not available. (see ?DateTimeClasses).
I would also like to point out the following from the help pages on timezones :
Note that except where replaced, the operation of time zones is an OS
service, and even where replaced a third-party database is used and
can be updated (see the section on ‘Time zone names’). Incorrect
results will never be an R issue, so please ensure that you have the
courtesy not to blame R for them.
The problem isn't R, but your Debian installation disregarding daylight saving time. You can solve this by configuring R with the option --with-internal-tzcode, so it uses its own timezone database. But this is generally not necessary if your Debian's timezone system is set up correctly. More info on how to configure R can be found on the help page ?timezones and in the Installation and Administration manual - appendix B.
The best way to solve this, is to make sure that your Debian installation deals with daylight saving time correctly. You can start with checking whether you have a correct version of the tzdata package.
There's a similar question on unix.stackexchange.com :
https://unix.stackexchange.com/questions/274878/system-disregards-daylight-saving-time

Datetime and Pytz Timezone .weekday() issue

I'm running into an issue when I'm trying to create a histogram of specific createdAt datetimes for orders. The issue is that even after created timezone aware datetimes, the .weekday() shows up as the same day, even though it should be a different time
The code I'm using to test this occurrence is as follows:
import datetime
import pytz
value = {
'createdAt': '2017-04-24T00:48:03+00:00'
}
created_at = datetime.datetime.strptime(value['createdAt'], '%Y-%m-%dT%H:%M:%S+00:00')
timezone = pytz.timezone('America/Los_Angeles')
created_at_naive = created_at
created_at_aware = timezone.localize(created_at_naive)
print(created_at_naive) # 2017-04-24 00:48:03
print(created_at_aware) # 2017-04-24 00:48:03-07:00
print(created_at_naive.weekday()) # 0 (Monday)
print(created_at_aware.weekday()) # 0 (should be Sunday)
The problem is that you need to actually change the datetime to the new timezone:
>>> timezone('UTC').localize(created_at)
datetime.datetime(2017, 4, 24, 0, 48, 3, tzinfo=<UTC>)
>>>timezone('UTC').localize(created_at).astimezone(timezone('America/Los_Angeles'))
datetime.datetime(2017, 4, 23, 17, 48, 3, tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)

Resources