Upload file size limit in web service

In order to reduce the load on the server (CPU/memory/network bandwidth) and avoid DDoS attacks, the file upload size of the client must be limited.

Commonly used megabytes and bytes correspondence table:
quote
  1 MB = 1048576 B
  2 MB = 2097152 B
  5 MB = 5242880 B
10 MB = 10485760 B
20 MB = 20971520 B
50 MB = 52428800 B
100 MB = 104857600 B
200 MB = 209715200 B


The general request processing flow is: Client -> Web Server -> Web Container -> Framework.
Therefore, the file size can be limited in any link.

(1) Client-side restrictions

Commonly used open source file upload components can limit the file size.

jQuery-File-Upload: maxFileSize
https://github.com/blueimp/jQuery-File-Upload/wiki/Options

plupload: max_file_size
https://github.com/moxiecode/plupload/wiki/Options#filters.max_file_size

Dropzone: maxFilesize
http://www.dropzonejs.com/#config-maxFilesize

FineUploader: sizeLimit
https://docs.fineuploader.com/features/validation.html

(2) Web server limit

Nginx: client_max_body_size (ngnix.conf)
http:/ /nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size
is 1MB by default, and returns 413 (Request Entity Too Large) directly after it exceeds. Set to 0 for unlimited.

server {

    client_max_body_size 1m;

    location /users/profile/edit/avatar {
        client_max_body_size 2m;
    }

    location /users/profile/edit/images {
        client_max_body_size 5m;
    }

}


Apache: LimitRequestBody (.htaccess)
https://httpd.apache.org/docs/2.4/en/mod/core.html#limitrequestbody
The default is 0 (unlimited), and a value from 0 to 2GB can be specified.

<VirtualHost *:8000>

    <Location />
        LimitRequestBody 1048576
    </Location>

    <Location /users/profile/edit/avatar>
        LimitRequestBody 2097152
    </Location>

    <Location /users/profile/edit/images>
        LimitRequestBody 5242880
    </Location>

</VirtualHost>


PHP:upload_max_filesize(php.ini)
quote
memory_limit = 40M
post_max_size = 32M
upload_max_filesize = 32M


(3) Web container limits

Tomcat connector: maxPostSize (server.xml)
http://tomcat.apache.org/tomcat-8.5-doc/config/ajp.html
http://tomcat.apache.org/tomcat-8.5 -doc/config/http.html
is 2MB by default. Set to 0 for unlimited.

<Connector port="8009"
           protocol="AJP/1.3"
           redirectPort="8443"
           maxPostSize="10485760" /> <-10MB

<Connector port="8080"
           protocol="HTTP/1.1"
           redirectPort="8443"
           maxPostSize="10485760"/> <-10MB


Servlet 3.0:multipart-config(web.xml)
<multipart-config>
    <location>/tmp</location>
    <max-file-size>5242880</max-file-size>
    <max-request-size>10485760</max-request-size>
    <file-size-threshold>32768</file-size-threshold>
</multipart-config>


or Servlet class annotation:
@MultipartConfig(
    location="/tmp",
    maxFileSize=5242880,
    maxRequestSize = 10485760,
    fileSizeThreshold=32768
)


(4) The framework restricts

SpringBoot settings:
#application.properties
spring.http.multipart.max-file-size=5MB
spring.http.multipart.max-request-size=10MB


Java configuration
    @Bean
    public MultipartConfigElement multipartConfigElement() {
        MultipartConfigFactory factory = new MultipartConfigFactory();
        factory.setMaxFileSize("5MB");
        factory.setMaxRequestSize("10MB");
        return factory.createMultipartConfig();
    }

StandardServletMultipartResolver is used by default, so the above definition is consistent with the definition of Servlet 3.0. CommonsMultipartResolver can also be used.

Apache Commons Upload settings:
    @Bean
    public CommonsMultipartResolver multipartResolver() {
        CommonsMultipartResolver cmr = new CommonsMultipartResolver();
        cmr.setMaxUploadSize(10485760);
        cmr.setMaxUploadSizePerFile(5242880);
        return cmr;
    }



After the file size is limited, how do I upload a very large file? This article is about how to download large files, which is similar to how to upload large files. The large files are divided into small chunks and uploaded one by one

. It needs to be supported on both the browser side and the server side.

If the browser
itself supports it, most open source components can do split upload, for
example, jQuery-File-Upload can specify the maxChunkSize parameter: https://github.com/blueimp/jQuery-File-Upload/wiki/Chunked- file-uploads
$(function () {
    $('#upload').fileupload({
        url: 'http://localhost:8080/upload',
        type: "POST",
        dataType: 'json',
        singleFileUploads: true,
        autoUpload: true
    });
    
    $('#uploadChunked').fileupload({
        url: 'http://localhost:8080/upload',
        type: "POST",
        dataType: 'json',
        singleFileUploads: true,
        autoUpload: true,
        maxChunkSize: 1048576
    });
});


When the server
receives file data, special processing is required.
- According to whether there is a Content-Range parameter in the HTTP header, it is divided into two different methods
- If it is a Chunk upload, the first time is to create a file, and then the content is appended to the file
@RestController
public class UploadController {

	private static final String FILE_DIR = "d:\\";
	private static final Pattern RANGE_PATTERN = Pattern.compile("bytes ([0-9]+)-([0-9]+)/([0-9]+)");

	@PostMapping(value = "/upload", headers = "!Content-Range")
	public Map<String, String> uploadMultipart(@RequestParam("file") final MultipartFile file, HttpServletResponse res) throws Exception {
		String fileName = file.getOriginalFilename();
		File source = new File(FILE_DIR + fileName);
		file.transferTo(source);

		Map<String, String> result = new HashMap<String, String>();
		result.put("name", FILE_DIR + fileName);
		return result;
	}

	@PostMapping(value = "/upload", headers = "Content-Range")
	public Map<String, String> uploadChunked(HttpServletRequest req, HttpServletResponse res) throws Exception {

		String contentRange = req.getHeader("Content-Range");
		String begin = "";
		Matches matches2 = RANGE_PATTERN.matches (contentRange);
		if (matcher2.matches()) {
			begin = matcher2.group(1);
		}

		MultipartHttpServletRequest multiReq = (MultipartHttpServletRequest) req;
		MultipartFile f = multiReq.getFile("file");
		String fileName = f.getOriginalFilename();

		if ("0".equals(begin)) {
			File source = new File(FILE_DIR + fileName);
			f.transferTo(source);
		} else {
			Path file = Paths.get(FILE_DIR, fileName);
			Files.write(file, f.getBytes(), StandardOpenOption.APPEND);
		}

		Map<String, String> result = new HashMap<String, String>();
		result.put("name", FILE_DIR + fileName);
		return result;
	}

}

Guess you like

Origin http://43.154.161.224:23101/article/api/json?id=326234553&siteId=291194637