So the documentation for Typesafe's Slick is very thin and its examples are for Play which doesn't help very much working in Eclipse.
I try to connect to an existing SQLite database on my system, which consists of one table "Maintenance_Request".
import slick.driver.SQLiteDriver.api._
import scala.concurrent.ExecutionContext.Implicits.global
object starter {
def main(args: Array[String]): Unit = {
val db = Database.forURL("jdbc:sqlite:/home/sq/workspace/dbFun/IOdb.db", driver = "org.sqlite.JDBC")
val action = sql"select CATEGORY from MAINTENANCE_REQUEST".as[(Int)]
db.run(action).foreach(println)
}
}
Starting the program doesn't give me any result though. Also if i change the path, like leaving out a character, so it is not a valid path, doesn't throw an error! So i don't have a clue what is working and what is not working.
Is there a way of knowing , if variable db is connected to a database?
Any way of knowing that Database.forURL worked or failed??
I suspect the reason you see no result is just that your main program is completing before the query has completed.
What I mean is, the type of db.run(action) will be Future[Seq[Int]]. Calling foreach on that does not block for a result. The behaviour of foreach on a Future is noted here:
It is important to note that calling the foreach combinator does not block to traverse the value once it becomes available. Instead, the function for the foreach gets asynchronously executed only if the future is completed successfully.
-- http://docs.scala-lang.org/overviews/core/futures.html
So what you can do is await the result:
import scala.concurrent.Await
import scala.concurrent.duration._
val result = Await.result(db.run(action), 2 seconds)
result.foreach(println)
...which will ensure you see the result printed before your program terminates.
In terms of errors connecting to a database...
in forUrl, if the connection string is not a valid recognized scheme, you'll see an exception such as java.sql.SQLException: No suitable driver
if you give a valid URL, but the wrong database name, that is going to depend on the database in question. For H2, and I suspect SQLite, you'll probably create a new empty database. There may be parameters you can pass in the JDBC URL to control this behaviour, but that will be driver/database specific.
Related
I am using sqlalchemy + asyncpg, and 'selectin' eager loading.
I have Person items that have one-to-many relationships with Friends.
I insert a Person into my database, with no related Friend entries. If in the same session I try and get that Person from the database, I can access their static (non-relationship) columns fine, but cannot access the friends relationship.
I think trying to access person.friends is triggering a lazy load, despite it being enforced previously as a selectin load. Why is this? How can I avoid it?
# Create the ORM model
class Person(Base):
__tablename__ = 'items'
id_ = Column(POSTGRES_UUID(as_uuid=True), primary_key=True)
name = Column(String(32))
friends = relationship('Friend', lazy='selectin')
# Create an instance
person_id = uuid4()
person = Person(id_=person_id, name='Alice') # Note that this Person's friends are not set
# Add to database
async with AsyncSession(engine, expire_on_commit=False) as session:
try:
session.begin()
session.add(person)
await session.commit()
except:
await session.rollback()
raise
# Get the added person from the database
created_person = await session.get(person, person_id)
print(created_person.id_) # Works fine
print(created_person.friends) # Raises error
Error:
sqlalchemy.exc.MissingGreenlet: greenlet_spawn has not been called; can't call await_() here.
Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/14/xd2s)
The solution is to use the populate_existing parameter in get:
populate_existing – causes the method to unconditionally emit a SQL query and refresh the object with the newly loaded data, regardless of whether or not the object is already present.
Replace
created_person = await session.get(person, person_id)
with
created_person = await session.get(person, person_id, populate_existing=True)
session.get documentation
See also: https://github.com/sqlalchemy/sqlalchemy/issues/7176
#theo-brown's answers goes straight to the point, but wanted to add some interesting information here.
Adding extra context on lazy loading and async SQLAlchemy:
When you fetch data using async SqlAlchemy, every model being queried spawns a coroutine. If you don't eager load your relationships, you'll end up with partially populated models.
Imagine this use case that I'm working on: I have a batch_job object, that relates to a batch_file and batch_job entries, all of which are tables in my database. When I don't eager load them, see what happens in the debugger:
The Traceback that I get when returning the object from an endpoint is this one:
greenlet_spawn has not been called; can't call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/14/xd2s)
The reason is that I didn't await these values, and that's what eager loading does for you in async sqlalchemy.
However, you might not have to eager load if you're working inside the application scope and you'll want to use these values later, and hence you could await them.
For those who are using the ORM, you could do it with the good old loading options:
results = await db_session.execute(select(YourModel).options(joinedload(YourModel.relationshipcolumn)).all()
In the samples for the Cosmos DB SQL API, there are a couple of uses of Database.ReadAsync() which don't seem to be doing anything useful. The remarks second in the method's documentation doesn't really indicate what it might be used for either.
What is the reason for using it in these cases? When would you typically use it?
ChangeFeed/Program.cs#L475 shows getting a database then calling ReadAsync to get another reference to the database
database = await client.GetDatabase(databaseId).ReadAsync();
await database.DeleteAsync();
which seems to be functionally the same as
database = client.GetDatabase(databaseId);
await database.DeleteAsync();
throwing the same exception if the database is not found.
and DatabaseManagement/Program.cs#L80-L83
DatabaseResponse readResponse = await database.ReadAsync();
Console.WriteLine($"\n3. Read a database: {readResponse.Resource.Id}");
await readResponse.Database.CreateContainerAsync("testContainer", "/pk");
which seems to be equivalent to:
Console.WriteLine($"\n3. Read a database: {database.Id}");
await database.CreateContainerAsync("testContainer", "/pk");
producing the same output and creating the container as before.
You are correct that those samples might need polishing, the main difference is:
GetDatabase just gets a proxy object, it does not mean the database actually exists. If you attempt an operation on a database that does not exist, for example, CreateContainer, it can fail with a 404.
ReadAsync will read the DatabaseProperties and allow you obtain any information from there and also would succeed if the database actually exists. Does that guarantee that if I call CreateContainer right away it will succeed? No, because the database could have been deleted right in the middle.
So in summary, ReadAsync is good if you want to get any of the DatabaseProperties or if you want to for some reason verify the database exists.
Most common scenarios would just use GetDatabase because you are probably attempting operations down the chain (like creating a container or executing item level operations in some container in that database).
Short Answer
Database.ReadAsync(...) is useful for reading database properties.
The Database object is useful for performing operations on the database, such as creating a container via Database.CreateContainerIfNotExistsAsync(...).
A bit more detail
The Microsoft Docs page for Database.ReadAsync is kind of confusing and not well written in my opinion:
The definition says:
Reads a DatabaseProperties from the Azure Cosmos service as an
asynchronous operation.
However, the example shows ReadAsync returning a DatabaseResponse object, not a DatabaseProperties object:
// Reads a Database resource where database_id is the ID property of the Database resource you wish to read.
Database database = this.cosmosClient.GetDatabase(database_id);
DatabaseResponse response = await database.ReadAsync();
It's only after a bit more digging that things become clearer. When you look at the documentation page for the DatabaseResponse Class it says the inheritance chain for DatabaseResponse is:
Inheritance: Object -> Response<DatabaseProperties> -> DatabaseResponse
If you then have a look at the Docs page for the Response<T> Class you'll see there is an implicit operator that converts Response<T> to T:
public static implicit operator T (Microsoft.Azure.Cosmos.Response<T> response);
So that means that even though the ReadAsync method returns a DatabaseResponse object, that is implicitly converted to a DatabaseProperties object (since DatabaseResponse inherits Response<DatabaseProperties>).
So Database.ReadAsync is useful for reading database properties.
The Docs page for Database.ReadAsync could have clearer about the implicit link between the DatabaseResponse object returned by the method and the DatabaseProperties object that it wraps.
I am trying to call an external API from an Excel on web. However, I am stuck on trying to get the result from the fetch call. I am even using the Office doc example to make sure
From an Excel, click on Automate to create a new script
async function main(workbook: ExcelScript.Workbook): Promise<void> {
let fetchResult = await fetch('https://jsonplaceholder.typicode.com/todos/1');
let json = await fetchResult.json();
}
I keep on getting the following message (at the fetchResult.json() call)
"Office Scripts cannot infer the data type of this variable or inferring it might result in unexpected errors. Please annotate the type of the variable to avoid this error. You can also use the Quick fix option provided in the editor to auto fill the type based on the usage. Quick Fix can be accessed by right clicking on the variable name and selecting Quick Fix link."
When running the Chrome inspector, the API request seems to be on hold "CAUTION: request is not finished yet"
PS: I am not the Office administrator and is not reachable right now, but hoping this is not a problem with my user or the Office account configuration
Any idea what the problem might be?
Thanks!
"any" types not being allowed in OfficeScript is by design. We think any types in general can lead to developer errors. I understand it can be hard to declare types – but these days most popular APIs provide you the interface (or d.ts) that you can use.
Secondly, there are tools such as https://quicktype.io/typescript where you can type in your sample JSON and it’ll give you the full interface which you can then declare in your code using interface keyword.
See this code for example: https://github.com/sumurthy/officescripts-projects/blob/main/API%20Calls/APICall.ts
You don’t need to declare all properties – only the ones you’ll use.
It’s more up-front work – but in the end the quality is better.
Adding an interface definition for the expected JSON type fixed the problem for me.
interface Todo {
userId: number;
id: number;
title: string;
completed: boolean
}
async function main(workbook: ExcelScript.Workbook): Promise<void> {
let fetchResult = await fetch('https://jsonplaceholder.typicode.com/todos/1');
let json: Todo = await fetchResult.json();
console.log(json);
}
You may need to define a different interface if the Web API you're calling returns different data structure.
I want to tell if an XML document has been constructed (e.g. using xdmp:unquote) or has been retrieved from a database. One method I have tried is to check the document-uri property
declare variable $doc as document-node() external;
if (fn:exists(fn:document-uri($doc))) then
'on database'
else
'in memory'
This seems to work well enough but I can't see anything in the MarkLogic documentation that guarantees this. Is this method reliable? Is there some other technique I should be using?
I think that behavior has been stable for a while. You could always check for the URI too, as long as you expect it to be from the current database:
xdmp:exists(fn:doc(fn:document-uri($doc)))
Or if you are in an update context and need ACID guarantees, use fn:exists.
The real test would be to try to call xdmp:node-replace or similar, and catch the expected error. Those node-level update functions do not work on constructed nodes. But that requires an update context, and might be tricky to implement in a robust way.
If your XML document is in-memeory, you can use in-mem-update API
import module namespace mem = "http://xqdev.com/in-mem-update" at "/MarkLogic/appservices/utils/in-mem-update.xqy";
If your XML document exists in your database you can use fn:exists() or fn:doc-available()
The real test of In-memory or In-Db is xdmp:node-replace .
If you are able to replace , update , delete a node then it is in database else if it throws exception then it's not in database.
Now there are two situation
1. your document is not created at all:
you can use fn:empty() to check if it is created or not.
2. Your document is created and it's in memory:
if fn:empty() returns false and xdmp:node-replace throws exception then it's in-memory
I am writing a DSL for expressing flow (original I know) in groovy. I would like to provide the user the ability to write functions that are stored and evaluated at certain points in the flow. Something like:
states {
"checkedState" {
onEnter {state->
//do some groovy things with state object
}
}
}
Now, I am pretty sure I could surround the closure in quotes and store that. But I would like to keep syntax highlighting and content assist if possible when editing these DSLs. I realize that the closure COULD reference artifacts from the surrounding flow definition which would no longer be valid when executing the closure in a different context, and I am fine with this. In reality I would like to use the closure syntax for a non-closure function definition.
tl;dr; I need to get the closure's code while evaluating the DSL so that it can be stored in the database and executed by a script host later.
I don't think there is a way to get a closure's source code, as this information is discarded during compilation. Perhaps you could try writing an AST transformation that would make closure's syntax tree available at runtime.
If all you care about is storing the closure in the database, and you don't need later access to the source code, you can try serializing it and storing the serialized form.
Closure implements Serializable, and after nulling its owner, thisObject and delegate attributes I was able to serialize it, but I'm getting ClassNotFoundException on deserialization.
def myClosure = {a, b -> a + b}
Closure.metaClass.setAttribute(myClosure, "owner", null)
Closure.metaClass.setAttribute(myClosure, "thisObject", null)
myClosure.delegate = null
def byteOS = new ByteArrayOutputStream()
new ObjectOutputStream(byteOS).writeObject(myClosure)
def serializedClosure = byteOS.toByteArray()
def input = new ObjectInputStream(new ByteArrayInputStream(serializedClosure))
def deserializedClosure = input.readObject() // throws CNFE
After some searching, I found Groovy Remote Control, a library created specifically to enable serializing closures and executing them later, possibly on a remote machine. Give it a try, maybe that's what you need.