phonenumbers.PhoneNumber as a FastAPI response_model field - fastapi

FastAPI supports having some (predefined) classes as pydantic model fields and have them be converted to JSON. For example datetime:
class MyModel(pydantic.BaseModel):
created_at: datetime.datetime
When used this model would convert datetime to/from str in the output/input JSON, when used as a response model or request body model, respectively.
I would like to have similar type safety for my own classes:
class MyModel(pydantic.BaseModel):
phone_number: phonenumbers.PhoneNumber
This can be made to work for request body models by using a custom validator but I also need MyModel to be convertible to JSON. Is this possible to achieve today? Note that I don't control the PhoneNumber class so the solution can't involve modifying that class.
Edit: the best I've come up with but still doesn't work:
def phone_number_validator(value: str) -> phonenumbers.PhoneNumber:
...
class MyModel(pydantic.BaseModel):
phone_number: phonenumbers.PhoneNumber
_validate_phone_number = pydantic.validator(
'phone_number', pre=True, allow_reuse=True)(phone_number_validator)
class Config:
arbitrary_types_allowed = True
json_encoders = {
phonenumbers.PhoneNumber: lambda p: phonenumbers.format_number(
p, phonenumbers.PhoneNumberFormat.E164),
}
This fails in FastAPI with:
fastapi.exceptions.FastAPIError: Invalid args for response field! Hint: check that <class 'phonenumbers.phonenumber.PhoneNumber'> is a valid pydantic field type

As you have already noticed, this is a bug in FastAPI. I just created a PR to fix it.
The arbitrary_types_allowed config directive is lost during the processing of the response model.
Until the PR is merged, you can use the workaround of monkey-patching the Pydantic BaseConfig like this:
from pydantic import BaseConfig
...
BaseConfig.arbitrary_types_allowed = True
# Your routes here:
...
But keep in mind that independent from this bug you might also need to adjust the JSON schema for the custom type, if you want the OpenAPI docs to work properly. Arbitrary types are not generally supported by the BaseModel.schema() method.
For that you can probably just inherit from phonenumbers.PhoneNumber and set a proper __modify_schema__ classmethod. See here for an example. Though I have not looked thoroughly into phonenumbers.
Check the example code in my PR text, if you want to see how you could implement validation and schema modification on your PhoneNumber subclass.
PS
Here is a full working example:
from __future__ import annotations
from typing import Union
from fastapi import FastAPI
from phonenumbers import PhoneNumber as _PhoneNumber
from phonenumbers import NumberParseException, PhoneNumberFormat
from phonenumbers import format_number, is_possible_number, parse
from pydantic import BaseModel, BaseConfig
class PhoneNumber(_PhoneNumber):
#classmethod
def __get_validators__(cls):
yield cls.validate
#classmethod
def validate(cls, v: Union[str, PhoneNumber]) -> PhoneNumber:
if isinstance(v, PhoneNumber):
return v
try:
number = parse(v, None)
except NumberParseException as ex:
raise ValueError(f'Invalid phone number: {v}') from ex
if not is_possible_number(number):
raise ValueError(f'Invalid phone number: {v}')
return number
#classmethod
def __modify_schema__(cls, field_schema: dict) -> None:
field_schema.update(
type="string",
# pattern='^SOMEPATTERN?$',
examples=["+49123456789"],
)
def json_encode(self) -> str:
return format_number(self, PhoneNumberFormat.E164)
class MyModel(BaseModel):
phone_number: PhoneNumber
class Config:
arbitrary_types_allowed = True
json_encoders = {
PhoneNumber: PhoneNumber.json_encode,
}
test_number = PhoneNumber(
country_code=49,
national_number=123456789
)
# Test:
obj = MyModel(phone_number=test_number)
obj_json = obj.json()
parsed_obj = MyModel.parse_raw(obj_json)
assert obj == parsed_obj
BaseConfig.arbitrary_types_allowed = True
api = FastAPI()
#api.get("/model/", response_model=MyModel)
def example_route():
return MyModel(phone_number=test_number)

Related

FastAPI pydantic response_model

FastAPI with uvicorn. So far i have everything working fine, now wanted to add extra validation with pydantic and response_model validation. My challange is, the response is a nested object, example:
{
"name": "always the same",
"value": ["ACL1", "ACL2", "ACL3"]
}
I have tried to:
class ACL(BaseModel):
name: str
#app.post(
"get/acls",
response_model=List[ACL]
)
def get_acls(credentials: HTTPBasicCredentials = Depends(security)):
But obviously it will fail. How to define correctly the response model so that it's correctly validated ? (and also visible in openapi documentation page ?)
Thanks,
You can use nested models in Pydantic and by by extension also in FastAPI. Like so:
class ACL(BaseModel):
name: str
class YourResponseModel(BaseModel):
name: str
value: list[ACL]
#app.post("/get/acls", response_model=YourResponseModel)
def get_acl():
return {"name": "always the same", "value": [{"name":"ACL1"}, {"name":"ACL2"}, {"name":"ACL3"}]}
If you don't want nested objects in your value list, then you have to build some logic to parse it. Below is an example of this:
class ACL(BaseModel):
name: str
class AnotherResponseModel(BaseModel):
name: str
value: list[str]
#app.post("/get/acls2", response_model=AnotherResponseModel)
def get_acl():
acls = [ACL(name="ACL1"), ACL(name="ACL2"), ACL(name="ACL3")]
return {"name": "always the same", "value": [acl.name for acl in acls] }
Please note, it is bad practice to mix up POST and GET requests (you are defining a POST endpoint, that will 'get' some ACLs for the end user). That is however unrelated to this question.

FastAPI RuntimeError: Use params or add_pagination

I'm writing my second project on FastAPI. And I got this error.
For example I have this code in my routers.users.py:
#router.get('/', response_model=Page[Users])
async def get_all_users(db: Session = Depends(get_db)):
return paginate(db.query(models.User).order_by(models.User.id))
And it works. It has fields limit and page in swagger documentation.
I tried to write the same for routers.recipes.py, but in this case I have no fields for pagination(limit, page) in swagger. Ok, I googled and found out that adding dependencies could help me. And now I see pagination parameters in swagger, but error is still the same.
routers.recipes:
#router.get('/', response_model=Page[PostRecipes], dependencies=[Depends(Params)])
async def get_all_recipes(db: Session = Depends(get_db)):
return paginate(db.query(models.Recipe).order_by(models.Recipe.id))
pagination:
class Params(BaseModel, AbstractParams):
page: int = Query(1, ge=1, description="Page number")
limit: int = Query(50, ge=1, le=100, description="Page size")
def to_raw_params(self) -> RawParams:
return RawParams(
limit=self.limit,
offset=self.limit * (self.page - 1),
)
class Page(BasePage[T], Generic[T]):
page: conint(ge=1) # type: ignore
limit: conint(ge=1) # type: ignore
__params_type__ = Params
#classmethod
def create(
cls,
items: Sequence[T],
total: int,
params: AbstractParams,
) -> Page[T]:
if not isinstance(params, Params):
raise ValueError("Page should be used with Params")
return cls(
total=total,
items=items,
page=params.page,
limit=params.limit,
)
__all__ = [
"Params",
"Page",
]
So, does anyone have ideas about it?
according to doc you have to specify default parameters,
your code should look like paginate(iterable, params)

Fastapi and Pydantic to build POST API: TypeError: Object of type is not JSON serializable

I have a problem with FastAPI and Pydantic.
I want to build a post api, program show this:
#router.post('/productRoute', response_model=SuccessCreate, status_code=status.HTTP_201_CREATED)
async def create_product_route(create: CreatePR):
query = ProductRouteModel.insert().values(
user_id=create.user_id,
route_id=create.route_id,
route_name=create.route_name,
head=create.head.dict(),
body=create.body,
route=create.route
)
await database.execute(query)
return {"status": "Successfully Created!"}
This is Pydantic class:
class RouteSchema(BaseModel):
id: str
next: Optional[List[str]]
prev: List[str]
class HeadSchema(BaseModel):
b1: str
b2: str
b3: str
class BodySchema(BaseModel):
a1: Optional[str]
a2: Optional[str]
class CreatePR(BaseModel):
user_id: str
route_id: str
route_name: str
head: HeadSchema
body: List[BodySchema]
route: List[RouteSchema]
Finally, this i want to post's parameter format:
{
user_id: "test1",
route_id: "route_1",
route_name: "route_name",
head: {...},
body: [{...}, {...}, ..., {...}],
route: [{...}, {...}, ..., {...}]
}
When I executed, I got TypeError: Object of type BodySchema is not JSON serializable.
How can I fix the program to normal operation?
Your code seems OK. I would not make a strong statement, but I suppose your post body is erroneous. Could you please verify whether your JSON format is correct or not. You could check it by using an online JSON editor (ex: https://jsonbeautifier.org/). Probable errors could be the usage of single quotes, missing/extra commas or even perhaps you forgot to put any quotes on your keys.

Airflow - predefine variables and connections in file

Is it possible to pre-define variables, connections etc. in a file so that they are loaded when Airflow starts up? Setting them through the UI is not great from a deployment perspective.
Cheers
Terry
I'm glad that someone asked this question. In fact since Airflow completely exposes the underlying SQLAlchemy models to end-user, programmatic manipulation (creation, updation & deletion) of all Airflow models, particularly those used to supply configs like Connection & Variable is possible.
It may not be very obvious, but the open-source nature of Airflow means there are no secrets: you just need to peek in harder. Particularly for these use-cases, I've always found the cli.py to be very useful reference point.
So here's the snippet I use to create all MySQL connections while setting up Airflow. The input file supplied is of JSON format with the given structure.
# all imports
import json
from typing import List, Dict, Any, Optional
from airflow.models import Connection
from airflow.settings import Session
from airflow.utils.db import provide_session
from sqlalchemy.orm import exc
# trigger method
def create_mysql_conns(file_path: str) -> None:
"""
Reads MySQL connection settings from a given JSON file and
persists it in Airflow's meta-db. If connection for same
db already exists, it is overwritten
:param file_path: Path to JSON file containing MySQL connection settings
:type file_path: str
:return: None
:type: None
"""
with open(file_path) as json_file:
json_data: List[Dict[str, Any]] = json.load(json_file)
for settings_dict in json_data:
db_name: str = settings_dict["db"]
conn_id: str = "mysql.{db_name}".format(db_name=db_name)
mysql_conn: Connection = Connection(conn_id=conn_id,
conn_type="mysql",
host=settings_dict["host"],
login=settings_dict["user"],
password=settings_dict["password"],
schema=db_name,
port=settings_dict.get("port", mysql_conn_description["port"]))
create_and_overwrite_conn(conn=mysql_conn)
# utility delete method
#provide_session
def delete_conn_if_exists(conn_id: str, session: Optional[Session] = None) -> bool:
# Code snippet borrowed from airflow.bin.cli.connections(..)
try:
to_delete: Connection = (session
.query(Connection)
.filter(Connection.conn_id == conn_id)
.one())
except exc.NoResultFound:
return False
except exc.MultipleResultsFound:
return False
else:
session.delete(to_delete)
session.commit()
return True
# utility overwrite method
#provide_session
def create_and_overwrite_conn(conn: Connection, session: Optional[Session] = None) -> None:
delete_conn_if_exists(conn_id=conn.conn_id)
session.add(conn)
session.commit()
input JSON file structure
[
{
"db": "db_1",
"host": "db_1.hostname.com",
"user": "db_1_user",
"password": "db_1_passwd"
},
{
"db": "db_2",
"host": "db_2.hostname.com",
"user": "db_2_user",
"password": "db_2_passwd"
}
]
Reference links
With code, how do you update an airflow variable?
Problem updating the connections in Airflow programatically
How to create, update and delete airflow variables without using the GUI?
Programmatically clear the state of airflow task instances
airflow/api/common/experimental/pool.py

Custom Adapter to support RocketPant with Rails

I am using rocket_pants gem to build backend API https://github.com/Sutto/rocket_pants
It have a specific format to output data:
{
"response":[
{"id":1,"title":"Object Title","description":"Object Description"},
{"id":1,"title":"Object Title","description":"Object Description"} ],
"count":2,
"pagination": {
"previous":null,
"next":null,
"current":1,
"per_page":30,
"count":2,
"pages":1}
}
I am using Batman.RailsStorage to persist models. But actions like MyApp.Model.get('all') works fine on the backend but they actually do not parse and load model objects.
Can you guide me how to configure StorageAdapter or write new one to handle such kind of data format?
You could try overriding the collectionJsonNamespace method (defined on Batman.RestStorage).
I see that it's used after a readAll operation to get records from the HTTP response.
For example:
class MyApp.RocketPantsStorage extends Batman.RailsStorage
collectionJsonNamespace: -> "response"
Then in your model
#= require ../rocket_pants_storage
# the storage adapter must be available at load time
class MyApp.SomeModel
#persist MyApp.RocketPantsStorage
Does that work?
With the same approach mentioned in answer from #rmosolgo I build paginator as well.
class MyApp.RocketPantsPaginator extends Batman.ModelPaginator
totalCountKey: "pagination.count"
loadItemsForOffsetAndLimit: (offset, limit) ->
params = #paramsForOffsetAndLimit(offset, limit)
params[k] = v for k,v of #params
#model.load params, (err, records, env) =>
if err?
#markAsFinishedLoading()
#fire('error', err)
else
response = new Batman.Object(env.response)
#set('totalCount', response.get(#totalCountKey));
#updateCache(#offsetFromParams(params), #limitFromParams(params), records)

Resources