I'm storing some RESTful api calls into a relational database using sqlalchemy, and I'm looking for a way to 'deserialize' or pythonify some or all of the incoming fields, for instance I might have a json object like
{
'id': 1,
'created_at': '2021-05-27T03:22:38Z',
'count': '3'
}
and I would like a way to "automatically" deserialize the data, similar to how djangorestframework serializers work where fields like "created_at" could be defined as datetime fields, and you could optionally cast 'count' as an integer, and run something like
...setup
# get the json from before as a dict
item = client.get_item()
# somehow serialize here
session = Session()
item_model = Item(**item_data[0])
session.add(item_model)
session.commit()
https://www.django-rest-framework.org/api-guide/serializers/
You have multiple well knowns module to perform serialization (independant of any framework):
Marshmallow: https://marshmallow.readthedocs.io/en/stable/
Typesystem (by the creator of DRF): https://github.com/encode/typesystem
You can also do your own serializer, based on DRF serializer code, if your use case is simple, or just perform a lookup of your dict fields that perform validation/transformation.
Related
Inside my database i'm using fields of reference type on some documents, like this:
I'm trying to insert these property on firestore
_, err = userReference.Set(context.Background(), models.User{
Jobs: []*firestore.DocumentRef{employeeReference},
})
On this case I used an array of *DocumentRef, but even if is a unique DocumentRef doesn't work, I also tried to insert as type DocumentRef instead of the pointer, and also doesn't work, my User type is like that:
type User struct {
Jobs []*firestore.DocumentRef `json:"jobs"`
}
There's something that I can do to insert this type of data from go? On Javascript SDK, I remember that is only to insert the DocumentReference type directly on object that is achieved, but I'm facing this problem with Golang.
I have no experience with Golang but here you can find many examples.
I wonder if something like the following can give you any hints:
_, err := userReference.Set(context.Background(), models.User{
Jobs: []*firestore.DocumentRef{client.Doc("/selfManagedEmployees/K4qhd5k1c...")}
})
In EFCore5, implicit tables are saved as Dictionary<TKey, object> sets, knows as Property Bag Entity Types. However, I cannot figure out how to create a LINQ query with a Where() clause that compiles for MySQL for such a property bag entity type.
For example, this code successfully retrieves the IQueryable reference to an intermediate table (generated by EFcore5's implicity many-to-many feature) given the ISkipNavigation:
ISkipNavigation nav = // some many-to-many relationship
IQueryable<Dictionary<string, object>> intermediateTable =
context.Set<Dictionary<string, object>>(nav.JoinEntityType.Name);
And this code successfully retrieves all the entries in the intermediate table:
List<Dictionary<string, object>> joins = await intermediateTable.ToListAsync();
In the resulting List, each Dictionary has just one key/value (representing the row).
However, I cannot figure out how to add a .Where() clause to the LINQ query which will compile:
joinTable.Where(d => d.Keys.First() == "foo").ToList();
joinTable.Where(d => d.Keys.Any(k => k == "foo")).ToList();
The error is:
Translation of member 'Keys' on entity type 'MachinePartMachineProfile (Dictionary<string, object>)' failed. This commonly occurs when the specified member is unmapped. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'
I do not wish to do client-side parsing for performance reasons (the join table will be to big).
I note that the type reported by the error is MachinePartMachineProfile (Dictionary<string, object>). Some investigation showed that these types are being generated based upon the static Model.DefaultPropertyBagType (which is a Dictionary<string, object>). But despite staring at the EFCore code base, I cannot discern how to correctly query such a default property bag type.
I am using MySQL as my database, if it is relevant.
You can index the dictionary directly, with knowledge of the column name.
Working example would be:
joinTable.Where(d => d[columnName] == "foo").ToList();
And for the sake of completeness, if you have an ISkipNavigation instance, you can infer these keys as follows:
string foreignKey = nav.ForeignKey.Properties.First().GetColumnBaseName();
string localKey = nav.Inverse.ForeignKey.Properties.First().GetColumnBaseName();
I have the following objects on cosmosdb
And my code with cosmonaut library its like this:
var pageTemplate = await pageTemplateStore.Query().FirstOrDefaultAsync(x => x.Id == id);
where id its either 0 or 1, but I am always getting null as a result
The problem is that you haven't decorated your object's Id field with [JsonProperty("id")] so the built in LINQ to SQL translator can't convert it. It can be found in Cosmonaut's documentation
However you should not query for documents using their id. This will perform a query instead of a read an it will cost way more RUs and more time. Use the FindAsync method instead which will do a direct read and will only cost 1RU.
I have a form whose one field is type IS_JSON
db.define_table('vmPowerOpsTable',
Field('launchId',label=T('Launch ID'),default =datetime.datetime.now().strftime("%d%m%y%H%M%S")),
Field('launchDate',label=T('Launched On'),default=datetime.datetime.now()),
Field('launchBy',label=T('Launched By'),default = auth.user.email if auth.user else "Anonymous"),
Field('inputJson','text',label=T('Input JSON*'),
requires = [IS_NOT_EMPTY(error_message='Input JSON is required'),IS_JSON(error_message='Invalid JSON')]),
migrate=True)
When the user submits this Form, this data is also simultaneously inserted to another table.
db.opStatus.insert(launchId=vmops_launchid,launchDate=vmops_launchdate
,launchBy=vmops_launchBy,opsType=operation_type,
opsData=vmops_inputJson,
statusDetail="Pending")
db.commit()
Now from the scheduler, I am trying to retrieve this data and make a POST request.
vm_power_opStatus_row_data = vm_power_opStatus_row.opsData
Note in the above step I am able to retrieve the data. (I inserted it in a DB and saw the field exactly matches what the user has entered.
Then from the scheduler, I do a POST.
power_response = requests.post(vm_power_op_url, json=vm_power_opStatus_row_data)
The POST request is handled by a function in my controller.
Controller Function:
#request.restful()
def vmPowerOperation():
response.view = 'generic.json'
si = None
def POST(*args, **vars):
jsonBody = request.vars
print "Debug 1"+ str(jsonBody) ##-> Here it returns blank in jsonBody.
But if I do the same request from Outside(POSTMAN client or even python request ) I get the desired result.
Is anything going wrong with the data type when I am trying to fetch it from the table.
power_response = requests.post(vm_power_op_url,
json=vm_power_opStatus_row_data)
It appears that vm_power_opStatus_row_data is already a JSON-encoded string. However, the json argument to requests.post() should be a Python object, not a string (requests will automatically encode the Python object to JSON and set the content type appropriately). So, the above should be:
power_response = requests.post(vm_power_op_url,
json=json.loads(vm_power_opStatus_row_data))
Alternatively, you can use the data argument and set the content type to JSON:
power_response = requests.post(vm_power_op_url,
data=vm_power_opStatus_row_data,
headers={'Content-Type': 'application/json')
Also, note that in your REST POST function, request.vars is already passed to the function as **vars, so within the function, you can simply refer to vars rather than request.vars.
Let me first say I have a solution to this problem, but I'm interested in knowing whether there is a better way, and whether I'm doing something wrong.
I have a table of objects on the front-end of a webapp, I need to asynchronously load some data for the objects as it is needed on a per-object basis. The server returns a JSON array containing the data for that object, and the data contains the object's key, so I can update the object on the front-end with its data. When there is no data, I just get an empty array, which unfortunately presents no way of updating the object, since I don't have the key to update it with. This can result in another query later, which is a waste of time/resources. I can't modify the server, is there a way to do this nicely?
My current solution is to just set the object's data to an empty array before sending the request, then just update when the result is received if the result is nonempty.
I was wondering if there is a better/more idiomatic way to do this.
For reference, I'm using Elm with PostgREST as the backend.
You can use currying and partial function application to indicate which object ID should be updated.
I'm assuming you have some code similar to this:
type Msg
= ...
| FetchData Int
| DataFetched [Data]
| DataFetchFail Http.Error
-- inside the update function
update msg model =
case msg of
...
FetchData id =
model ! [ Task.perform DataFetchFail DataFetched (Http.post ...) ]
If you define your DataFetched constructor to include the ID as the first parameter, you can use partial application to include the ID for future lookup, regardless of what the server returns.
Here's the same code chunks with this idea:
type Msg
= ...
| FetchData Int
| DataFetched Int [Data]
| DataFetchFail Http.Error
-- inside the update function
update msg model =
case msg of
...
FetchData id =
model ! [ Task.perform DataFetchFail (DataFetched id) (Http.post ...) ]
You could also add the ID to the Fail message for more fine-grained error messages.