Front-end file upload practice and back-end processing - file upload in blocks

File upload is one of the common features in modern web applications. In this blog, we'll explore a simple but complete front-end file upload implementation, while providing a back-end example that demonstrates how to handle uploaded files. We will use JavaScript as the front-end language combined with Node.js as the back-end environment. let's start!

Front-end file upload

Let's focus on the front end first. We'll use HTML, CSS, and JavaScript to create a simple file upload form, then pass the file to a backend server via AJAX. Here we assume that you already have basic front-end development knowledge.

1. HTML structure

First, we create an HTML structure that includes a file upload input box, an upload button, and elements for displaying upload progress and information:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>文件上传示例</title>
</head>
<body>
  <input type="file" id="videoUploader" accept="video/mp4, video/ogg">
  <button id="uploadBtn">上传</button>
  <p id="uploadInfo"></p>
  <progress id="uploadProgress" value="0" max="100"></progress>
</body>
</html>

2. CSS styles (optional)

You can add some CSS styles to beautify the page as needed.

3. JavaScript logic

Next, we'll use JavaScript to implement the file upload logic. We start with a configuration file, then implement the upload function and related helper functions:

// config.js
// 导出一些常量，包括上传信息、允许上传的文件类型、块大小和上传接口的URL
// 这些常量将在前端和后端代码中使用
const BASE_URL = 'http://localhost:8000/';

export const UPLOAD_INFO = {
  'NO_FILE': '请先选择文件',
  'INVALID_TYPE': '不支持该类型文件上传',
  'UPLOAD_FAILED': '上传失败',
  'UPLOAD_SUCCESS': '上传成功'
}

export const ALLOWED_TYPE = {
  'video/mp4': 'mp4',
  'video/ogg': 'ogg'
}

export const CHUNK_SIZE = 64 * 1024;

export const API = {
  UPLOAD_VIDEO: BASE_URL + 'upload_video'
}

// index.js
// 在闭包内部，定义一个立即执行的匿名函数，接受document作为参数
((doc) => {
  // 获取页面上的一些元素
  const oProgress = doc.querySelector('#uploadProgress'); // 上传进度条
  const oUploader = doc.querySelector('#videoUploader'); // 文件上传输入框
  const oInfo = doc.querySelector('#uploadInfo'); // 用于显示上传信息
  const oBtn = doc.querySelector('#uploadBtn'); // 上传按钮

  let uploadedSize = 0; // 已上传的文件大小，用于控制分块上传

  // 初始化函数
  const init = () => {
    bindEvent(); // 绑定事件
  }

  // 绑定事件函数
  function bindEvent() {
    oBtn.addEventListener('click', uploadVideo, false); // 点击上传按钮触发上传函数
  }

  // 异步上传函数
  async function uploadVideo() {
    // 获取文件对象
    const { files: [ file ] } = oUploader;
    
    if (!file) {
      // 检查是否选择了文件，如果没有则显示提示信息
      oInfo.innerText = UPLOAD_INFO['NO_FILE'];
      return;
    }

    if (!ALLOWED_TYPE[file.type]) {
      // 检查文件类型是否被允许，如果不允许则显示提示信息
      oInfo.innerText = UPLOAD_INFO['INVALID_TYPE'];
      return;
    }

    // 获取文件的一些基本信息
    const { name, type, size } = file;
    const fileName = new Date().getTime() + '_' + name;
    let uploadedResult = null;
    oProgress.max = size;
    oInfo.innerText = '';

    while (uploadedSize < size) {
      // 使用slice方法将文件切割成小块，实现分块上传
      const fileChunk = file.slice(uploadedSize, uploadedSize + CHUNK_SIZE);
      // 创建FormData对象，用于传递文件块和相关信息
      const formData = createFormData({
        name, 
        type,
        size,
        fileName,
        uploadedSize,
        file: fileChunk
      });

      try {
        // 使用axios库发送POST请求，将文件块上传到后端
        uploadedResult = await axios.post(API.UPLOAD_VIDEO, formData);
      } catch (e) {
        // 捕获异常，如果上传失败，则显示提示信息
        oInfo.innerText = `${ UPLOAD_INFO['UPLOAD_FAILED'] }（${ e.message }）`;
        return; 
      }

      // 更新已上传的文件大小和进度条
      uploadedSize += fileChunk.size;
      oProgress.value = uploadedSize;
    }

    // 上传成功后，显示上传成功的提示信息，并将文件输入框置空
    oInfo.innerText = UPLOAD_INFO['UPLOAD_SUCCESS'];
    oUploader.value = null;
    // 根据后端返回的视频URL，创建一个视频元素并展示在页面上
    createVideo(uploadedResult.data.video_url);
  }

  // 创建FormData函数，将文件块和相关信息添加到FormData对象中
  function createFormData ({
    name, 
    type,
    size,
    fileName,
    uploadedSize,
    file
  }) {
    const fd = new FormData();

    fd.append('name', name);
    fd.append('type', type);
    fd.append('size', size);
    fd.append('fileName', fileName);
    fd.append('uploadedSize', uploadedSize);
    fd.append('file', file);

    return fd;
  }

  // 创建视频元素函数，用于在页面上展示上传成功的视频
  function createVideo(src) {
    const oVideo = document.createElement('video');
    oVideo.controls = true;
    oVideo.width = '500';
    oVideo.src = src;
    document.body.appendChild(oVideo);
  }

  init(); // 初始化，执行绑定事件函数
})(document);

Import dependencies:
- config.js: Export some constants, including upload information, file types allowed to be uploaded, block size and the URL of the upload interface.
- axios: A library for making HTTP requests to upload file chunks to the backend.
Create an anonymous function that executes immediately:
- Accepts documentas an argument, and uses docto represent documentthe object.
- This approach avoids global variable pollution and ensures that code runs in a separate scope.
Get elements on the page:
- oProgress: An element representing the upload progress bar <progress>.
- oUploader: An element representing the file upload input box <input type="file">.
- oInfo: An element used to display upload information <p>.
- oBtn: The element representing the upload button <button>.
Initialization function:
- Call bindEvent()the function, which is used to bind the event handler when the upload button is clicked.
Binding event function:
- Using addEventListener()the method, add a click event listener for the upload button.
- When the upload button is clicked, uploadVideo()the function will be triggered to upload the file.
Asynchronous upload function uploadVideo():
- Get the file object , which is obtained filethrough the file upload input box .oUploader
- Checks if a file is selected, displays a prompt if not, and returns.
- Check whether the file type is allowed to be uploaded, if not, display a prompt message and return.
- Get some basic information about a file, such as file name, file type, and file size.
- Using whilea loop, upload the file in chunks:
  - Use file.slice()the method to chop the file into small chunks (chunk size CHUNK_SIZEis defined by a constant).
  - Create FormDataan object into which to add file chunks and related information for passing to the backend.
  - Using axios.post()the method, upload chunks of the file to the URL specified by the backend ( API.UPLOAD_VIDEOdefined by a constant).
  - If the upload fails, catch the exception and display a prompt message, and return.
  - Update the uploaded file size and the value of the progress bar.
- After the upload is complete, a prompt message indicating that the upload is successful is displayed, and the value of the file input box is set to blank.
- According to the video URL returned by the backend, call createVideo()the function to create a video element on the page and display the successfully uploaded video.
Create FormDatafunction createFormData():
- Adds file chunks and related information to FormDataan object for passing to the backend.
Create video element function createVideo():
- It is used to create a video element on the page, and set the video URL to the video URL returned by the backend after the upload is successful.
Initialization function init():
- Execute bindEvent()the function to bind the upload button and the event handler together.
Finally, init()the entire front-end code is started by calling a function.

We include axiosthe library in the front-end code, which is a common tool for making HTTP requests. In a real project, you need to make sure to import the library in the HTML file axios, or use other libraries with similar functions.

In the above code, we first define some constants, such as file types allowed to be uploaded, block size, etc. Then we capture the click event of the "upload" button through event monitoring and execute the upload function. In the upload function, we cut the file into multiple small pieces by uploading in pieces, and send them to the backend one by one for processing. During the upload process, we will also update the upload progress bar and display upload information in real time.

Backend file processing

Now let's focus on the backend processing part. We'll use Node.js and Express to set up a simple backend server that receives file chunks from the frontend and merges them into complete files.

1. Install dependencies

First, create a file in the project directory package.json, then run the following command to install dependencies:

The above code uses the Express framework to build a simple server and set the route for file upload. We can /upload_videoupload file chunks through the interface, and the backend saves the file chunks on the server side according to the uploaded information until all chunks are received and merged into a complete file.

npm init -y
npm install express body-parser express-fileupload

2. Backend code

Next, we create a server.jsfile and write the backend code:

const express = require('express');
const bodyParser = require('body-parser');
const uploader = require('express-fileupload');
const { extname, resolve } = require('path');
const { existsSync, appendFileSync, writeFileSync } = require('fs');

const app = express();
const PORT = 8000;

// 设置中间件，解析请求的JSON数据和文件上传
app.use(bodyParser.urlencoded({ extended: true }));
app.use(bodyParser.json());
app.use(uploader());

// 设置静态资源目录，用于存放上传的临时文件块
app.use('/', express.static('upload_temp'));

// 允许跨域访问
app.all('*', (req, res, next) => {
  res.header('Access-Control-Allow-origin', '*');
  res.header('Access-Control-Allow-Methods', 'POST,GET');
  next();
});

// 处理文件上传的POST请求，对应的URL为/upload_video
app.post('/upload_video', (req, res) => {
  // 从请求体中获取传递的文件信息和文件块
  const { name, type, size, fileName, uploadedSize } = req.body;
  const { file } = req.files;

  if (!file) {
    // 检查是否传递了文件块，如果没有则返回错误信息
    res.send({
      code: 1001,
      msg: 'No file uploaded'
    });
    return;
  }

  if (!ALLOWED_TYPE[type]) {
    // 检查文件类型是否被允许，如果不允许则返回错误信息
    res.send({
      code: 1002,
      msg: 'The type is not allowed for uploading.'
    });
    return;
  }

  const filename = fileName + extname(name);
  const filePath = resolve(__dirname, './upload_temp/' + filename);

  if (uploadedSize !== '0') {
    if (!existsSync(filePath)) {
      // 检查文件是否存在，如果不存在则返回错误信息
      res.send({
        code: 1003,
        msg: 'No file exists'
      });
      return;
    }

    // 文件已存在，则将当前文件块追加到已有的文件中
    appendFileSync(filePath, file.data);

    // 返回成功信息和视频URL
    res.send({
      code: 0,
      msg: 'Appended',
      video_url: 'http://localhost:8000/' + filename
    });

    return;
  }
  
  // 第一个文件块，创建一个新文件并写入当前块的数据
  writeFileSync(filePath, file.data);

  // 返回成功信息
  res.send({
    code: 0,
    msg: 'File is created'
  });
});

// 启动服务，监听指定端口
app.listen(PORT, () => {
  console.log('Server is running on ' + PORT);
});

Import dependencies:
- express: Express framework for building web servers.
- body-parser: Parse the JSON data in the request body.
- express-fileupload: Handles file upload requests.
- path: Node.js built-in module for handling file paths.
- fs: Node.js built-in module for file system operations.

Create an Express application:

const app = express();
const PORT = 8000;

Use middleware:
- bodyParserMiddleware is used to parse JSON data and form data in the request body.
- uploaderMiddleware is used to handle file upload requests.
- Set the static resource directory '/upload_temp'to store uploaded temporary file blocks.
Set cross-domain access: Use app.all()the method to set the response headers that allow cross-domain requests.
Handle file upload requests:
- Use app.post('/upload_video', ...)the definition to handle POST requests for file uploads, the corresponding URL is /upload_video.
- Obtain the passed file information and file blocks from the request body, including file name, file type, file size, upload start position of file blocks, etc.
- Checks whether the file block exists, and returns an error message if it does not exist.
- Checks if the file type is allowed, and returns an error message if not.
- Generate a new file name based on the file name and file type, and calculate the storage path of the file.
- If not the first file block, append the current file block to the existing file.
- If it is the first file block, create a new file and write the current block's data.
- Returns a success message, including the URL of the uploaded video.
Start the service: Use app.listen()the method to start the server, listening on the specified port (8000 in this case).

run project

Now, we have finished coding the front end and back end. Next, we need to run the project locally to test the file upload functionality.

1. Create a temporary upload directory

upload_tempCreate a folder named .py at the root of your project to hold uploaded temporary chunks of files.

2. Start the server

Run the following command to start the backend server:

node server.js

3. Start the front end

Put the front-end code (HTML, CSS, and JavaScript) in a folder, and then run an HTTP server under the folder (it’s okay to use npm and yarn, npm install——npm run dev/yarn——yarn dev) , such as using http-server:

npx http-server

Now, visit the front-end page (usually http://localhost:8080) through the browser. Select a supported video file (.mp4 or .ogv format), click the upload button, and you will see an upload progress bar showing the upload progress. After the upload is complete, the page will display the message that the upload was successful, and display the uploaded video file on the page.

conclusion

In this blog, we learned how to implement a simple front-end file upload function, and combined Node.js and Express to build a back-end server to handle file uploads. In actual projects, you can expand and optimize the file upload function according to your needs. I hope this blog can help you understand the basic principles and practical methods of file upload. thanks for reading!

If you still have questions about front-end file upload, or are interested in other technical topics, welcome to join the discussion group or pay attention!