[Agile software engineering team blog] Technical Specifications

project	content
Spring 2020 Computer Software Engineering Institute (Roger Ren Jian)	Park class blog blog
Work requirements	Technical Specifications
Our objective in this course is	Exercise themselves in teamwork
In particular aspects of the job which help us achieve our goals	Technical development of functional specifications and instructions

Overview & technology stack

Outline

Our project continues the Luo Jidong seniors graduated from the design, to further improve the design and GitLab based software engineering practice teaching platform. Individual project work for the soft course, a knot All In One solution for project teams and project management.

Technology stack

GitLab related

OAuth Applications gitlab : provide login services.

GitLab API

Runner CI + gitlab gitlab : every code change automatically compile the test, and the results are sent back to GitLab.
Hook + WebHook System : monitor various events (such as project creation, code, check-in, etc.), send a report to the specified URL, provide the raw data for the students and teachers for analysis.

rear end

Ruby on Rails

Haml : rendering Web pages.
Bundler : providing gem background application dependency management.
Nginx : Rails to provide reverse proxy service.
Webpacker : Vue.js and integrated front-end frame MVVM Rails.

database

Sqlite3 / PostgreSQL : providing development / production dual-database support, by switching gem freedom.

front end

Vue.js

UI Element : provide front-end UI component library, the drawing page.
Echarts : providing visual diagrams support constructed burn-down chart like.
The Yarn : provides package management tool.
WebPACK : front-end modular management.
Babel : compiling JavaScript code.

product design

Architecture

Macro architecture

Macro architecture design

Interface Specifications (tentative)

#	Request method	Request path	Routing point	use
1	`post`	`/system`	`system#index`	添加system-wide webhook
2	`post`	`/webhook`	`webhook#index`	Add to each project webhook
3	`post`	`/report`	`report#index`	Automatic acceptance of test results report Interface
4	`get`	`/login`	`sessions#login`	OAuth authentication login
5	`get`	`/oauth/callback`	`sessions#login`	OAuth authentication login application
6	`get`	`:issues : all`	`issues#all_issues`	The total amount of issues
7	`get`	`:projects : kanban`	`boards#index`	Kanban project
8	`post`	`:projects : uploads`	`uploads#index`	upload files
9	`get`	`:projects : labels`	`projects#labels`	Gets project label
10	`get`	`:projects : members`	`projects#members`	Gets project members
11	`get`	`:milestones : kanban`	`boards#index`	Gets progress chart
12	`get`	`:milestones : burndown`	`burndown#index`	Get burndown
13	`post`	`:classroom : feedback`	`auto_test_projects#feedback`	The applicant list
14	`post`	`:classroom : trigger`	`auto_test_projects#trigger`	Approves the application
15	`get`	`:team_projects : raw`	`blogs#show_rawn`	Display raw files
16	`get`	`:team_projects : insight`	`insight#show`	Browse action
17	`get`	`:blogs : raw`	`blogs#show_raw`	Show blog raw files
18	`get`	`member: join`	`classrooms#join`	Students join
19	`get`	`member: exit`	`classrooms#exit`	Students exit

analysis

Abstract and decoupling

Our system is mainly based on Vue.js other front-end frame and the back-end Ruby on Rails framework. This section, we will before, in our system to achieve high cohesion and low coupling in accordance with the modular abstraction by the order from the bottom up.

Decoupling database and back-end framework
- Ruby on Rails framework database corresponding entities are converted into the object in Ruby, using a database description language (DDL) database manipulation language (DML) to create and manipulate a database conversion and to Ruby object entity corresponding to the example of a method It calls.
- For each type of common database, Ruby on Rails framework are able to provide a more comprehensive and reliable support. Only you need to Gemfilespecify the project can rely on gem file, for example:
```
group :development do
  gem 'sqlite3'
end
group :production do
  gem 'pg'
end
```
  As can be seen, in our development environment, convenient and practical for a lightweight database Sqlite3, will be used in a production environment PostgreSQL.
- Provide direct SQL database query language functions in Ruby on Rails framework, but we do not use this method to avoid differences between different SQL database language processing system and cause problems occur when changing the database.
- Based on the above considerations, we realized the full decoupling of the database and back-end framework, so that we develop and production environments, we can achieve more freedom to switch. For example, in the project directory config/database.yml, you can freely configure the database development and production environments in use:
```
development:
  <<: *default
  adapter: sqlite3
  database: db/development.sqlite3

production:
  <<: *default
  adapter: postgresql
  encoding: unicode
  database: teach
  username: deploy
  password: <%= ENV['TEACH_DATABASE_PASSWORD'] %>
```
  After switching to a new database, just run rails db:migrateautomatically deploy database entities, flexibility is very strong.
Internal backend decoupling
- Ruby on Rails framework using MVC design pattern, that is, Models, Views, Controllers. Wherein, Models and databases for the corresponding entity, for routing Controllers response to user requests, processing for receiving the Views of Controllers, and ultimately render a page presented to the user.
- First, the user issues a Request, according to the set of routing tables (in config/routes.rb), passes the request to the corresponding controller as a parameter, the controller does not have to care about how the user is to call its methods can be invoked when needed as long as a method in which routing give the right to appropriate. Models need to interact with the corresponding in. Then, as long as the rendering information needed to pass variables to the corresponding page views Views can, without concern for how to render a view to achieve a decoupling of the back-end and front end.
The rear end of the front end of the decoupling
- As mentioned above, the Views section receives only the parameters transmitted by the Controllers, and based on this rendering parameter corresponding page. Controllers do not care what Views to render the page. This design to achieve a good separation of interface and implementation.
- Using HTML pages with embedded Ruby statements usually ( *.html.erb), using haml (HTML abstraction markup language) in our project, aimed at making the front page code more simple and elegant. We use Vue.js and ElementUI as the front frame.

Model structure of the entire system as shown in FIG.

Separation platform to interact with the curriculum of the GitLab

On an analysis of the main course platform (ie the main part of our project) to interact with the separate internal. Since our projects are integrated in GitLab in, so this section we focus on interacting with the separation between the project and GitLab platform.

Webhook

Webhook is "user-defined HTTP callbacks" which is triggered by certain events, including code push, create or modify the Issue and so on. After Webhook fall within the scope of the project, good Webhook configure callback URL in an integrated set of projects, the code push or modify Issues such as events occur every time, GitLab the URL will initiate a POST request to send relevant data to the URL (such as sending data commit this push when push event triggered). Software engineering practice teaching platform to achieve student paper and commit Issues tracking information through the registration Webhook, thereby forming a contribution report.

System Hook

Unlike Webhook, GitLab System Hook is GitLab the global level Hook, when the project to create, update, delete and update users to create, delete, Group create, update, System Hook will be called when deleted, way to call the registered data URL to send the current events (such as sending project data when the project is created). Software engineering practice teaching platform Sign System Hook By tracking project creation, adding Webhook.

Interact with the separation and analysis

Our platform is mainly to get information from and interact GitLab by the above Webhook and System Hook achieved. In addition, we GitLab front-end code has been modified a small place, making it possible to see the entrance of our software engineering platform in GitLab the Dashboard panel. In this way, we will GitLab two parts platform and production platform for the software engineering courses call the relationship between modules, two flat level module API call each other to provide each other, to achieve a loose decoupling between modules.

Adaptability of massive data

Adaptability massive data, in our project focuses on two aspects. On the one hand is static user information, on the other hand is a dynamic user interaction with the system.

Static user information

Here we look at some reference information online. Considering that our system is in line with the use of teaching software engineering, so our subscriber growth usually semester units, according to our previous blog analysis, our subscriber growth is relatively slow and not too early large (in the order of about 1000), therefore this part is not what we want to focus on consideration. Here, we collect information online from finishing, combined with our analysis of the project, as shown below.

Database separate read and write
- This mechanism is suitable for large data IO, and there is a case of reading performance bottlenecks. For example GitLab we use itself is based on Ruby on Rails framework to achieve, Ruby on Rails and agile development for the design, however, one drawback is that if large amount of data corresponding speed will be slower, for example, we pull from GitLab warehouse when students take the code while pulling too much can lead to read and write performance bottlenecks.
- Set a primary database and multiple slave database, the main library is responsible for a write operation, a read operation from the library responsible for, read and share the pressure off.
Distributed server
- Introducing a distributed server including GitLab distribution platform, etc., further enhanced parallelism.
- Load balancing is managed by Nginx or other solutions have been used Nginx reverse proxy in our system.

Dynamic user interaction

In our project, in order to be able to automate and facilitate the evaluation of the students were automatically deployed, we introduced GitLab CI, to help students achieve their continuous integration project, where we mainly use the GitLab Runner. Go GitLab Runner is based on the language development of open source projects, and it is used in conjunction GitLab CI, CI is used to run the task, and the result is sent back to the task CI GitLab. Since GitLab Runner can be run on multiple servers, as long as the configuration GitLab where when you create Runner IP address can be, so this design is for our students at the same time a large number of deployment projects or to ensure adequate time for evaluation Runner, improving parallelism, Stuck reduce user waiting time.

Information packages and hidden

Ruby on Rails

Our back-end based on Ruby on Rails framework. Wherein the package with the hidden information is mainly achieved through the package and access control class. Ruby is a highly object oriented language, one of its design principle known as "Everything Is an Object." As mentioned earlier, we package information database entity object instance to Ruby on Rails framework, access is controlled by the member variables and methods of object instances, the package can be realized for information hidden and the underlying database. Ruby on Rails framework and its application of MVC design pattern provides a great help to encapsulate and hide information.

GitLab

Our platform is integrated in the course of GitLab. GitLab design has been very mature, our platform is not the relationship between the internal details GitLab work, but only open by GitLab API interface to interact with it, so as to realize the package and hidden GitLab internal information.

Vue.js

Vue is a user interface used to build the progressive frame . The other frame difference is large, Vue is designed to be applied from the bottom up layer by layer. Vue core library focus only on the view layer, is not only easy to use but also easy to integrate third-party libraries or existing project. On the other hand, when used in conjunction with a modern tool chain and various support libraries, Vue also fully capable of providing drive for complex one-page application.

- From Vue.js official Chinese version Tutorials

Our front-end framework based primarily on Vue.js, Vue.js framework concept is a very important component, different functions can be encapsulated in the different components, and can provide interactive interface to call other components. Meanwhile, in addition we can not help but want to use CSS Cascading Style Sheets and JavaScript on the front end. In the course we will try to avoid for the style settings, such as in the HTML page divspecified tag style, or set a buttonlabel onclickfor a method of a JS file, and so on. Instead, we will document packages in strict CSS and JS, try to ensure the cleanliness of the HTML page.

flexibility

On the one hand, as mentioned before, our project due to the use of the Ruby on Rails framework to achieve a decoupling database framework, it has a strong flexibility in the replacement of the database. If you need to use different databases, as long as the Gemfileincrease in the corresponding gem package dependency database, then perform bundle installautomatically install all dependent gem package, using rails db:migratedatabase migration to.
On the other hand, we have the flexibility to deploy the project. We used Webpacker. Webpacker the Rails application use, management and packaging Javascript easier, facilitate the development and debugging of front-end components. Our project uses Webpacker to integrate front-end MVVM framework Vue.js and Rails. At the same time we also provide users with local and deployment scripts and docker mirrored on the server, to achieve true one-click deployment out of the box.

Error Handling

Request address error
- Illegal request
  - That is because they do not address the legitimate user to modify the Request which led to the emergence of an illegal address.
  - We Ruby on Rails framework routes.rbin the final set the default route. Once all previous requests and address have not been matched, it will go through this default route, corresponding to the Request Error Handling Controller for processing, for example, provide an error message, and then redirect the user to access their own personal home page and this message rendering.
- Unauthorized access
  - That is because the user intends to use the routing behavior of other users to access information or privileges.
  - We in the corresponding Controller associated with increased access control statements to control the behavior, such as:
```
before_action :correct_user, only: [:list_orders, :destroy, :month_income, :year_income,:monthly, :month_average]
```
    Here we write a method (in the above example correct_user), identity identification and authentication information based on the user's Session and their corresponding records stored in the database, only after certification lawful only allow access onlymethod after, otherwise, its done error handling is similar to illegal request, giving its message to inform its no access to redirect it to your home page, and render this information.
Database stored error
- Here mainly refers to the complete any form and content of the form will correspond to a data record stored procedure entity to the database.
- Here we mainly through the improvement of checks on the legality of the entry form to try to make it a precaution, on this basis, we restrict access to the database, as far as possible front-end data entry via the API provided under normal circumstances, to avoid direct operation of the database, check the legality of bypassing the front-end database into which led to the illegal data.

to sum up

The above analysis, our project is based on a front-end Vue.js, back-end based on Ruby on Rails, while incorporating a comprehensive project in GitLab in. We fully taken into account in the design of decoupling between the inter-module level and, at the same time for both flexibility and security were considered and design. We will continue to improve its iterative and in the development of the project, which will eventually create a platform for the software engineering course can really put into teaching applications.