Importing tables from web to google sheets - web-scraping

I would like to import series of tables from a page which only shows a table at a time, rest can be selected by dropdown. Is there a way I can add values of table without importing hundreds of tables and processing? For example I would like to import all tables of this page
http://www.demoscope.ru/weekly/ssp/emp_lan_97_uezd.php?reg=1 to *reg=949 . First 2 columns are the same in all tables but columns 3-5 are different. I would like to get a final table adding columns 3-5 in all tables. Is it possible? Or how to import multiple tables at once? Thanks in advance.

try like this:
={ IMPORTHTML(A1&1; "table"; 7)\
QUERY(IMPORTHTML(A1&2; "table"; 7); "select Col3,Col4,Col5")\
QUERY(IMPORTHTML(A1&3; "table"; 7); "select Col3,Col4,Col5")\
QUERY(IMPORTHTML(A1&4; "table"; 7); "select Col3,Col4,Col5")\
QUERY(IMPORTHTML(A1&5; "table"; 7); "select Col3,Col4,Col5")\
QUERY(IMPORTHTML(A1&6; "table"; 7); "select Col3,Col4,Col5")\
QUERY(IMPORTHTML(A1&7; "table"; 7); "select Col3,Col4,Col5")}
where A1:
http://www.demoscope.ru/weekly/ssp/emp_lan_97_uezd.php?reg=

Related

Using JSON_EXTRACT in WHERE clause using Laminas-Db

I have a mysql table with a json datatype used to store json_encoded data:
id|author_id|date_inserted|comments
---------------------------------------------
1|32|2022-04-01|{"id": "343","some":"data"}
2|15|2022-02-15|{"id": "24","some":"data"}
3|24|2022-05-22|{"id": "995","some":"data"}
Using laminas-db, how is it possible to use JSON_EXTRACT in a where clause? For instance, when using a tablegateway, i would like to update the json data of a row using something like the following:
$this->tableGateway->update($data, [JSON_EXTRACT('comments', '$.id') = 24]);
I was able to achieve it with the following:
$commentId = 24;
$where = new Where();
$where->addPredicate(new Expression('JSON_EXTRACT(comments, "$.id") = ?', $commentId));
return $this->tableGateway->update($data, $where);

Android SQLITE Insert into table with values coming from a subquery

In my db-driven app I need to perform insert into queries in which the value for one or more field comes from a subquery.
The insert into statement may look like the following example:
INSERT INTO MyTable (field_1, field_2)
VALUES('value for field 1', (SELECT field_x FROM AnotherTable WHERE ...))
At present I am doing it manually building the query:
String MyQuery = "INSERT INTO mytable (field_1, field_2)
VALUES('value for field 1', (SELECT field_x FROM AnotherTable WHERE ...))"; // Of course my query is far more complex and is built in several steps but the concept is safe, I end up with a SQL String
SQLiteDatabase= db = getWritableDatabase();
db.execSQL(MyQuery); // And it works flawlessy as it was a Swiss Clock
What i would like to do instead is:
SQLiteDatabase db = getWritableDatabase();
ContentValues values = new ContentValues();
values.put("field_1", "value for field 1");
values.put("field_2", ThisIsAQuery("(SELECT field_x FROM AnotherTable WHERE ...)"));
db.insert("MyTable", null, values);
db.close();
Where the fake method ThisIsAQuery(...) is the missing part, something that should tell the query builder that "SELECT.." is not a value but a query that should be embedded in the insert statement.
Is there a way to achieve this?
The whole point of the ContentValues container is to be able to safely use strings without interpreting them as SQL commands.
It is not possible to use subqueries with insert(). The only way to get a value from another table is by executing a separate query; in this case, ThisIsAQuery() would be stringForQuery() or longForQuery().

JDBC - SQLITE Select to variable

I am trying to run a query / select statement and save it in a variable. I know how to get something specific from a specific column but not from counting rows.
This is working as I getting MYID specifically.
ResultSet MYIDrs = stmtCFG.executeQuery( "SELECT rowid, MYID from MYINDEX order by rowid desc limit 1;" );
MYID = MYIDrs.getString("MYID");
Now I am trying to count the rows that works in SQLite client but not in the jdbc as I can't figure out what to request.
this is what I have but is not resulting in what I am expecting.
ResultSet FILE_COUNTrs = stmtCFG.executeQuery( "SELECT count(*) from TABLE where MYID = '"+MYID+"';");
FILE_COUNT = FILE_COUNTrs.getString(?????);
problem or question is: What do I put in the ????? as I already tried everything.
I am expecting to see a number.
I am really sorry I found what I was looking for by assigning a name TOTAL
This is my code and it works...
ResultSet FILE_COUNTrs = stmtCFG.executeQuery( "SELECT count(*) AS TOTAL from TABLE where MYID = '"+MYID+"';");
FILE_COUNT = FILE_COUNTrs.getString("TOTAL");
You use wrong data type. COUNT(*) returns Integer type, Not String.
You can do like this without assigning a label for COUNT(*)
int FILE_COUNT = FILE_COUNTrs.getInt(1); // 1: is the column index of COUNT(*)

Conditionally parameterize SQLite query in AIR

How can I conditionally parametrize a SQLite database in AIR?
For example this query:
//selectedID is the ID I want to select
query.text = "select * from table where id=#ID";
query.parameters['#ID']=selectedID;
But I want the where statement to appear only if selectedID is greater than 0.
What I would normally do is:
query.text = "select * from table"+(selectedID>0?" where id="+selectedID:'');
However, I read on the LiveDocs performance-wise it is better to use parameters.
Is it possible to parametrize a whole statement or that's only possible for values?
Or maybe this is good-enough:
query.text = "select * from table"+(selectedID>0?" where id=#ID":'');
query.parameters['#ID']=selectedID;
if (selectedID > 0)
{
query.text = .....
query.parameters['#ID'] = ...
}
else
{
query.text = .....
{

Flex: sqlite last_insert_rowid over multiple insert calls

I have file with multiple SQL statements in it to be executed.
INSERT INTO reports (a,b,c) VALUES (1,2,3);
INSERT INTO units (report_id, e, f, g) VALUES ( (SELECT last_insert_rowid() FROM reports), 4, 5, 6);
INSERT INTO elements (report_id, h, i, j) VALUES ( (SELECT last_insert_rowid() FROM reports), 7, 8, 9);
The FROM reports section of the sub-selection does nothing.
What ends up happening is:
A row is inserted into reports and the reports.id field is autoincremented
A row is inserted into units with report_id being equal to the reports id
A row is inserted into elements with report_id being equal to units.id of the last row inserted
This is works as described in the sqlite documentation.
My issue is that I want all the queries subsequent to the report insert to use report.id.
Is there anyway I can get this to work on the database end without having to resort to a solution in as3?
There is a way to do this, but it is in AS3 using parameters.
What is done is instead of using the SELECT last_insert_row() function in each call, replace it with a parameter.
INSERT INTO elements (report_id, h, i, j) VALUES (#id, 7, 8, 9);
Now in my code I have to split the file into an array so that each individual queries is process separately (this is how AS3 implements sqlite's API).
var sqlArray:Array = sql.split(";\n\n");
Now what I do is execute the first statement for importing the report itself.
statement.text = sqlArray[0];
statement.execute();
Now the fun part. You need to get the id back out. So we run another query.
statement.text = "SELECT last_insert_rowid() as ID";
statement.execute();
var id:int = statement.getResult().data[0].id;
Now we can loop through the rest of the queries using the id as a parameter.
for(var i:int = 1; i < sqlArray.length - 1; i++) {
/**
* start at 1 because we already inserted the report
* end at length -1 because our last entry is empty because of how split works on our data
**/
statement.text = sqlArray[i];
statement.parameters['#ID'] = id;
statement.execute();
}
This is a little more complicated, but not much and it ends up working.
Everything rolled together into a single function (omitting a lot of class overhead) would be:
function importFromSQLString(sql:String):void {
try{
connection.begin();
var sqlArray:Array = sql.split(";\n\n");
statement.text = sqlArray[0];
statement.execute();
statement.text = "SELECT last_insert_rowid() as ID";
statement.execute();
var id:int = statement.getResult().data[0].id;
for(var i:int = 1; i < sqlArray.length - 1; i++) {
statement.text = sqlArray[i];
statement.parameters['#ID'] = id;
statement.execute();
}
connection.commit();
statement.clearParameters();
} catch (e:Error) {
connection.rollback(); //cleanup if there was a failure
}
}

Resources