Difficulties, processes and new developments in translating video hard subtitles

Hard subtitles usually refer to the text embedded in the video, which cannot be stripped by editing and playback software. If you want to extract and translate video subtitles, and finally synthesize video, you will probably need to use OCR, video restoration, translation, sentence simplification, video layout calculation, audio and video and other technologies. With the popularity of AI artificial intelligence in 2023, is there any product that can use these technologies comprehensively to directly extract the hard subtitles in the video and perform translation and synthesis software? Today, the editor will analyze the problems here and recommend a product that can really solve the translation problem of video hard subtitles.

The whole article is divided into the following parts

1. Definition of Video Hard Subtitles

2. Technical Principles and Difficulties of Video Hard Subtitle Translation

3. Translation effect of video hard subtitles

4. Tutorial for translating video hard subtitles

5. How to modify translated hard subtitles?

6. Adjustment method for inconsistent style and inaccurate translation after hard subtitle extraction

7. Which languages ​​are supported for translating video hard subtitles?

1. Definition of video hard subtitles

Hard subtitles are also called embedded subtitles, embedded subtitles, built-in subtitles, etc. Usually, the text of the subtitles is the subtitles that have been embedded in the movie. The text of this kind of subtitle is no longer text, but an image. It does not have a separate subtitle file, and users cannot use editing tools or playback tools to delete this option. These subtitles cannot be changed or completely removed.

2. Principles and difficulties of video hard subtitle translation technology

The technical process of extracting and translating hard subtitles of videos is as follows:

1. Video parsing and subtitle extraction: Use video parsing tools or open source libraries to extract subtitle files from videos. Hard subtitles are usually saved as pixel information in video files, and subtitles cannot be turned off or hidden. Send video files to OCR recognition engine or API to convert pixel information into text information.

2. Recognize text with the help of OCR technology: OCR technology can recognize text information in subtitles and then convert it into text form. The OCR engine will create a subtitle data set to identify possible special characters, such as punctuation marks, special symbols, font colors, and capital letters in subtitles. The difficulty here is style extraction.

3. Translated subtitles: Use language models such as ChatGPT for automatic translation. The machine translation API can be used to pass the extracted subtitle text to the translation engine and return the translation result to the application. The difficulty here is translation accuracy.

4. Replacement and integration of subtitles: Embed translated subtitles into the video's timeline and generate new subtitle files. (srt, ass and other formats). The difficulty here is to restore the video and remove the original hard subtitles.

5. Matching of audio files to subtitles: Use audio processing tools to synchronize the subtitles in the video with the audio files, keeping them consistent and eliminating any timing discrepancies. Difficulties are space and layout calculations, multi-font synthesis.

6. Generate translated video files: Reassemble synchronized audio and video files and output translated video files.

The entire technical process can be automated, and the controllable component is the process of natural language processing through OCR and ChatGPT API.

3. The effect of translating video hard subtitles

The following is the actual effect of the automatic translation software of Guishou Editing

GhostCut uses OCR to identify the "subtitles" in the original video and generate translated text. At the same time, it uses AI-video restoration to remove the original video subtitles, and then pastes the translated subtitles back to the position of the original subtitles, and keeps the original subtitles as much as possible. The size, color, plate type, etc. Compared with using a variety of editing and translation software, the features of Guishou Editing video translation are: fully automatic, one-click to remove the original text, and retain the original video text format.

4. The operation process of translating video hard subtitles

It only supports operations on the mobile phone, and the specific operation steps are as follows:

1. Open the chat application, search for the applet "Ghost Hand Editing" to enter;

2. Click [Translate Video Text], upload the video in the mobile phone album, select the original language type in the video [Chinese or English], and select the target translation language type [English or Chinese];

3. Check the options, click Submit, and wait a few minutes for your video to be translated.

4. Check the work in the work list and download it.

5. How to modify the translated hard subtitles?

Video subtitle translation now supports the adjustment of the translated text, if you think the translation

of

Inaccurate or mistranslated scene text can be adjusted in the following ways:

1. Click on the work in the applet, there is an option "Adjust video translation"

2. After clicking, you can view new and old works, and you can edit subtitles in batches at the same time.

3. After entering, you can see the editing subtitle area, click to enter and you can adjust the copywriting.

When adjusting the copy, pay attention to:

1. The style will not change;

2. Pay attention to the length of the text, if it is too long, it will automatically wrap;

3. You can also manually change the line in the translated area;

Tips:

If you find that the styles of two consecutive lines are inconsistent, you can delete one line and concentrate the copywriting on one line for processing.

edit search image

6. Adjustment method for inconsistent style and inaccurate translation after hard subtitle extraction

There may be many problems in the automatic translation video, including the subtitle style of the same line is separated, the translation is inaccurate, and the styles of different pictures are inconsistent. For the style and translation problems of the same picture, you can have the following tips:

1. Suppose the style of the original video is as follows

2. Problems after automatic extraction and translation: inaccurate translation and inconsistent font size

3. The solution, in the subtitle editor,

• Merge multi-line text (line 1 and line 2 merge)

• Delete redundant lines; (delete line 2)

• Manually break the line in the merged translation copy, as shown in the figure below

4. The final effect

Is not it simple?

Which languages ​​are supported for translating hard subtitles in videos?

Original language in the video: Chinese/English, etc.

Translated target languages: Chinese/English/Japanese/Indonesian/Portuguese/French/Spanish/Hindi/Arabic/Vietnamese/Thai/German/Russian/Turkish/Italian/Vietnamese

Well, this is the relevant technical support and product process of hard subtitle translation compiled by the editor for you. Welcome everyone to experience it. I have written so much, is it worth your careful attention?