This article is shared from Huawei Cloud Community "Field-based ORM implementation in Python without using metaclasses", author: Lemony Hug.
Simple ORM implementation without using metaclasses
In Python, ORM (Object-Relational Mapping) is a technology that converts the mapping relationship between objects and databases, making it more convenient to operate the database in an object-oriented manner. Usually, we use metaclasses to implement ORM, but this article will introduce a simple ORM implementation that does not use metaclasses.
Field class
First, we define a Field class to represent the fields in the database table. This class contains information such as the name and type of the field, and supports some comparison operations to facilitate subsequent construction of query conditions.
class Field:
def __init__(self, **kwargs):
self.name = kwargs.get('name')
self.column_type = kwargs.get('column_type')
def __eq__(self, other):
return Compare(self, '=', other)
# 其他比较操作略...
Compare class
In order to build query conditions, we introduced a Compare class to represent the comparison relationship between fields. It can support chain operations and build complex query conditions.
class Compare:
def __init__(self, left: Field, operation: str, right: Any):
self.condition = f'`{left.name}` {operation} "{right}"'
def __or__(self, other: "Compare"):
self.condition = f'({self.condition}) OR ({other.condition})'
return self
def __and__(self, other: "Compare"):
self.condition = f'({self.condition}) AND ({other.condition})'
return self
Model class
Next, we defineModel the class that represents the table in the database. This class defines the fields of the table through instances of theFieldclass and provides methods for inserting data.
class Model:
def __init__(self, **kwargs):
_meta = self.get_class_meta()
for k, v in kwargs.items():
if k in _meta:
self.__dict__[k] = v
@classmethod
def get_class_meta(cls) -> Dict:
if hasattr(cls, '_meta'):
return cls.__dict__['_meta']
_meta = {}
for k, v in cls.__dict__.items():
if isinstance(v, Field):
if v.name is None:
v.name = k
name = v.name
_meta[k] = (name, v)
table = cls.__dict__.get('__table__')
table = cls.__name__ if table is None else table
_meta['__table__'] = table
setattr(cls, '_meta', _meta)
return _meta
def insert(self):
_meta = self.get_class_meta()
column_li = []
val_li = []
for k, v in self.__dict__.items():
field_tuple = _meta.get(k)
if field_tuple:
column, field = field_tuple
column_li.append(column)
val = str(v) if field.column_type == 'INT' else f'"{str(v)}"'
val_li.append(val)
sql = f'INSERT INTO {_meta["__table__"]} ({",".join(column_li)}) VALUES ({",".join(val_li)});'
print(sql)
Query class
Finally, we implemented theQuery class for building database queries. This class supports chain calls and can set query conditions, sorting, etc.
class Query:
def __init__(self, cls: Model):
self._model = cls
self._order_columns = None
self._desc = ''
self._meta = self._model.get_class_meta()
self._compare = None
self.sql = ''
def _get(self) -> str:
sql = ''
if self._compare:
sql += f' WHERE {self._compare.condition}'
if self._order_columns:
sql += f' ORDER BY {self._order_columns}'
sql += f' {self._desc}'
return sql
def get(self, *args: Field) -> List[Model]:
sql = self._get()
table = self._meta['__table__']
column_li = []
if len(args) > 0:
for field in args:
column_li.append(f'`{field.name}`')
else:
for v in self._meta.values():
if type(v) == tuple and isinstance(v[1], Field):
column_li.append(f'`{v[0]}`')
columns = ",".join(column_li)
sql = f'SELECT {columns} FROM {table} {sql}'
self.sql = sql
print(self.sql)
def order_by(self, columns: Union[List, str], desc: bool = False) -> "Query":
if isinstance(columns, str):
self._order_columns = f'`{columns}`'
elif isinstance(columns, list):
self._order_columns = ','.join([f'`{x}`' for x in columns])
self._desc = 'DESC' if desc else ''
return self
def where(self, compare: "Compare") -> "Query":
self._compare = compare
return self
Example usage
Now, we can define a model class and use this simple ORM implementation for data manipulation.
class User(Model):
name = Field()
age = Field()
# 插入数据
user = User(name='Tom', age=24)
user.insert()
# 构建查询条件并查询数据
User.query().where((User.name == 'Tom') & (User.age >= 20)).order_by('age').get()
In this way, we have completed a simple ORM implementation that does not use metaclasses. Although the code structure is simpler than using metaclasses, in actual applications, it is important to choose an appropriate implementation method based on project requirements and team agreement.
We have introduced a simple ORM implementation based on Python that does not rely on metaclasses. In this part, we'll continue exploring this implementation, taking a deeper look at query building and more complex usage.
Extended query capabilities
Our query function is still relatively simple. In order to better support complex queries, we can add more query methods and conditions.
Support LIMIT and OFFSET
class Query:
# ...
def limit(self, num: int) -> "Query":
self.sql += f' LIMIT {num}'
return self
def offset(self, num: int) -> "Query":
self.sql += f' OFFSET {num}'
return self
Support GROUP BY and HAVING
class Query:
# ...
def group_by(self, columns: Union[List, str]) -> "Query":
if isinstance(columns, str):
columns = [columns]
self.sql += f' GROUP BY {",".join([f"`{x}`" for x in columns])}'
return self
def having(self, condition: Compare) -> "Query":
self.sql += f' HAVING {condition.condition}'
return self
Example usage
class User(Model):
name = Field()
age = Field()
# Insert data
user = User(name='Tom', age=24)
user.insert()
# Construct query conditions and query data
query = User.query().where((User.name == 'Tom') & (User.age >= 20)).order_by('age'). limit(1).offset(0)
query.get(User.name, User.age) # Only query the specified fields
# More complex queries
query = User.query().group_by('age').having((User.age > 20) & (User.age < 30)).order_by('age 39;).limit(10).offset(0)
query.get(User.age, User.count(User.name)) # Query the number of users aged between 20 and 30
By introducing additional query capabilities, we make this simple ORM implementation more powerful and flexible.
Summarize
In this series of articles, we implemented a simple Python ORM without using metaclasses. We define classes to represent database fields, classes to represent database tables, and classes for building and executing queries. Through this implementation, we can easily perform data operations and build flexible query conditions without having to deeply understand the concept of metaclasses. Field Model Query
However, this simple ORM still has some limitations, such as not supporting features such as complex table associations. In actual projects, the choice to use metaclass ORM implementation or other mature ORM frameworks depends on the needs of the project and the team's technology selection. I hope this implementation can provide you with a different way of thinking and prompt more thinking and discussion.
Click to follow and learn about Huawei Cloud’s new technologies as soon as possible~
Tang Xiaoou, founder of SenseTime, passed away at the age of 55 In 2023, PHP stagnated Wi-Fi 7 will be fully available in early 2024 Debut, 5 times faster than Wi-Fi 6 Hongmeng system is about to become independent, and many universities have set up “Hongmeng classes” Zhihui Jun’s startup company refinances , the amount exceeds 600 million yuan, and the pre-money valuation is 3.5 billion yuan Quark Browser PC version starts internal testing AI code assistant is popular, and programming language rankings are all There's nothing you can do Mate 60 Pro's 5G modem and radio frequency technology are far ahead MariaDB splits SkySQL and is established as an independent company Xiaomi responds to Yu Chengdong’s “keel pivot” plagiarism statement from Huawei