NewLife.XCode is a 10-year history of open-source middleware, data, support nfx / netcore, the team new life (2002 to 2019) developed and maintained so far, hereinafter referred to as XCode.
Throughout this series will be a substantial connection with example code and run logs in-depth analysis, it contains many years of development experience in which the representative of ten billion big data in real-time computing projects.
Open Source Address: https://github.com/NewLifeX/X (beg star, 864+)
XCode hyperemia model, the entity class included in a large number of convenient operation, including the various types of target data import and export!
Json serialization
The moment the most common data communication format is certainly no doubt Json, solid objects can be transferred directly to the interface Json as a return value.
See in general a Role entity object below Json serialization (toJSON) and Json deserialization (ToJsonEntity) routines and effects.
Above, Role json string can be converted to solid objects by ToJson, and the string to another through ToJsonEntity Role entity object.
Before XCode v9.0, base class entity with Entity directly ToJson / FromJson. Later, because Json too common, and to a global extension methods.
Json As the most common data transmission format (no one), ToJson direct extension Object, there are three parameters: whether the new line indent, whether writing a null value, whether camelCase
The examples above ToJson (true, false, false) expressed using line feed indentation (better readability), not write null (null 0 and an integer string, etc.), NA camelCasing.
Xml Serialization
As a former King of Xml data transmission format, there is still a considerable part of the old interface.
ToXml / ToXmlEntity also a global extension method, in which the control parameters can be serialized to normal or Xml Xml properties.
From the results, Xml readability is very good, but a large space, generally bigger than Json length.
Binary Serialization
XCode trick is serialized binary serialization, can make solid objects and binary data to one another, smaller, faster!
The same sequence of physical objects, only 39 bytes, is much less than the Xml and Json, and faster (no string dividing operation).
No method of rapid entity object ToBinary like, but need to be converted to IAccessor interface, and then into a serial data stream Write, or Read data stream.
Data flow is a good thing, the most commonly used memory stream, file streams, network flow, stream encryption, compression stream, and so on and so on.
File stream FileStream, you can make solid objects saved as a file, or the file is loaded into solid objects, (help file cache data entity?);
Network flow NetworkStream, allowing transmission in the network entity object;
IAccessor access interface, any object can be achieved to the data stream read access.
NewLife.Redis priority support IAccessor interface, so when the XCode entity object into the Redis, uses a streamlined and efficient binary format, rather than the default Json format .
Entity class may be defined from a binary serialization behavior by overriding OnWrite.
/// <Summary> binary sequence of a data stream </ Summary> /// <param name = "Stream"> stream </ param> /// <param name = "context"> context </ param> / // <param name = "extend"> whether serialization extended attribute </ param> protected Virtual Boolean OnWrite (Stream Stream, Object context, Boolean extend) { var bn = context as Binary; if (bn == null) bn = new Binary { Stream = stream, EncodeInt = true }; var fs = extend ? Meta.AllFields : Meta.Fields; foreach (var fi in fs) { bn.Write(this[fi.Name], fi.Type); } return true; }
As seen from the above code, Binary Binary Serialization just call to the respective data fields written to the stream.
See here, you are not a lot of ideas can make a big fight? Do not worry, this is just an appetizer, there are better.
See the example, binary serialization biggest drawback is unreadable! Generally only see one part of the string.
Entity list is stored
Binary Serialization single entity subject only suitable for simple applications, such as writing Redis, actual project-level applications, and more to the list of entities based.
List of entities to extend storage method provides:
- Write, is written into the data stream entity list
- Read, Load entity list from the data stream
- SaveFile, save the entity list to a file (used as a data cache)
- LoadFile, load the file from the list of entities
The above example demonstrates the role list has four objects if the read and write data stream and file.
Friend to ask, whether the list of entities with the right features to read and write files, localized caching some data tables, even if the database is down, is still able to continue to provide services?
The obvious answer is yes, and I often do it in practice.
Data analysis often need to use dozens of hundreds of servers for analysis calculations, you must configure the data and common data base disposable loaded into memory when the application starts, for example, tens of thousands of rows of data networks;
If dozens of nodes simultaneously start the data network requests to the database, the database will be overwhelmed very slow, affecting computing applications load and may even cause an error exit;
This scenario may be loaded once, save the data entity list to a local file, then the timing (10 minutes) to update;
The next time you start the direct use locally cached data, greatly enhance the application launch speed, and reduce the burden on the database;
Csv Import Export
Csv format commonly used in the Office Excel data transfer, database and data import and export and so on.
XCode support one million export! Internal CsvFile supports streaming read and write, while construction while writing to the file, rather than a one-time write again generated good memory!
Cube Excel export, Csv export, Json export, Xml export and other functions, by XCode achieve!
Tutorial Series
NewLife.XCode tutorial series [2019 Edition]
- CRUD entry . Quick show usage code to configure the connection string
- Data model file . Create a form fields and indexes, names and data types specifications, recommended field (time, user, IP)
- Detailed entity class . Data traffic class type, a generic base class, the interface
- Feature set . The connection string, the debugging, the SQL log, log slow, parametric, execution timeout. Code profile settings, the connection string partial set
- Reverse engineering . Data automatically create database tables
- Data initialization . InitData write initialization data
- Senior additions and deletions . Overload interception, increment field, validation! Valid solid model (time, the user, the IP)
- Dirty data . How to produce, how to use
- Delta Sigma . High concurrency statistics
- Transaction processing . Single and multi-table form, the different connections, a variety of writing
- Extended attributes . Multi-table association, Map map
- Advanced query . Complex conditions, paging, custom extensions FieldItem, check the total number of records, check summary statistics
- Data caching layer . Sql cache update mechanism
- Entity cache . Full table finishing cache update mechanism
- Object cache . Dictionary cache for user data such as more scenes.
- Ten billion performance . Refining field, the index is complete, reasonable inquiries, make full use of cache
- Physical plant . Metadata handler general
- Role permissions . Membership
- Import and export . Xml, Json, binary, network or file
- Points table and warehouses . Split common logic
- Advanced statistics . Statistics polymerization, group statistics
- Batch write . Bulk insert batch Upsert, asynchronous save
- Entity queue . Write-level caching to improve performance.
- Backup synchronization . Data backup, data recovery, data synchronization
- Data services . RPC interface to provide services, remote execution of queries, such as SQLite Online
- Big data analytics . ETL extraction, scheduling calculation processing, persistence results