The web version of Photoshop is here, what front-end technologies are used?

After years of hard work by Adobe engineers and close cooperation with browser vendors such as Chrome, through the support of WebAssembly + Emscripten, Web Components + Lit, Service Workers + Workbox and new Web API, the Web version of Photoshop has finally been launched recently ( photoshop.adobe.com), which has significant implications in enabling highly complex and graphics-intensive software to run in the browser!

This article will take a look at the Web capabilities used by Photoshop, its performance optimizations, and possible future development directions.

Vision: Use Photoshop in the browser

Adobe's vision is to bring Photoshop to the browser so that more users can easily use it for image editing and graphic design. Photoshop has been the gold standard for image editing and graphic design for the past few decades, but it only runs on the desktop. Now, by porting it to the browser, a whole new world opens up.

Photoshop on the web promises ubiquitous, frictionless access. Users only need to open a browser and immediately start editing and collaborating with Photoshop without installing any software. Moreover, because the Web is a cross-platform operating environment, it can shield the differences in the underlying operating systems, allowing Photoshop to interact with users on different platforms.

Plus, sharing workflows becomes even more convenient with the link feature. Photoshop documents can be accessed directly via URL. This way, creators can easily send links to collaborators for more convenient collaboration.

But realizing this vision faces significant technical challenges, requiring a rethinking of how intensive applications like Photoshop run on the Web.

Use new web capabilities

A number of new Web platform capabilities have emerged in recent years that can be standardized and implemented to eventually make applications like Photoshop possible. Adobe engineers innovatively leverage several key next-generation APIs.

Use OPFS to achieve high-performance local file access

Photoshop operations involve reading and writing PSD files that can be very large. This requires efficient access to the local file system, and the new Origin Private File System API (OPFS) provides a fast, origin-specific virtual file system.

Origin Private File System (OPFS) is a Web API that provides fast and secure access to local file systems. It allows web applications to read and write local files natively without exposing the files directly to the web environment. OPFS enables secure access to files by running a local agent in the browser and using specific file system paths.

const opfsRoot = await navigator.storage.getDirectory();

Use OPFS to quickly create, read, write, and delete files. For example:

// 创建文件

const file = await opfsRoot.getFileHandle('image.psd', { create: true });



// 获取读写句柄

const handle = await file.createSyncAccessHandle();



// 写入内容



handle.write(buffer);



// 读取内容

handle.read(buffer);



// 删除文件

await file.remove();

In order to achieve absolutely fast synchronization operations, you can use Web Workers to obtain it FileSystemSyncAccessHandle.

This native, high-performance file system is critical in enabling the demanding file workflows required for Photoshop in the browser. It provides fast and reliable file reading and writing capabilities, allowing Photoshop to process large files more efficiently. This optimized file system brings users a smoother image editing and processing experience.

Unleashing the power of WebAssembly

WebAssembly is one of the key elements in re-implementing Photoshop's computationally intensive graphics processing in JavaScript. To port existing C/C++ code bases to JavaScript, Adobe uses the Emscripten compiler to generate WebAssembly module code.

In this process, WebAssembly has several crucial capabilities:

SIMD : Use SIMD vector instructions to speed up pixel operations and filtering.
Exception Handling : C++ exceptions are used extensively in Photoshop’s code base.
Streaming instantiation : Since Photoshop's WASM module is over 80MB in size, streaming compilation is required.
Debugging : The WebAssembly debugging support provided by Chrome in DevTools is very useful
Threads : Photoshop uses worker threads to perform tasks in parallel, such as processing blocks of images:

// 线程函数

void* tileProcessor(void* data) {

    // 处理图像块数据

    return NULL;

}



// 启动工作线程

pthread_t thread1, thread2;

pthread_create(&thread1, NULL, tileProcessor, NULL);

pthread_create(&thread2, NULL, tileProcessor, NULL);



// 等待线程结束

pthread_join(thread1, NULL);

pthread_join(thread2, NULL);

Utilizing P3 wide color gamut

The P3 color gamut is wider than the sRGB color gamut and can display a wider range of colors. However, for a long time, sRGB has been the only color gamut standard on the Web, and other wider color gamuts such as P3 have not been widely adopted.

Photoshop leverages new color()functions and the Canvas API to take full advantage of the vividness of the P3 color gamut, resulting in more accurate color rendering. By using these features, Photoshop can better display the richer, more vivid colors contained in the P3 color gamut.

color: color(display-p3 1 0.5 0)

Web Components provide UI flexibility

Photoshop is part of the Adobe Creative Cloud ecosystem. By using the standardized Web Components strategy built on Lit[1], UI consistency between applications can be achieved.

Lit is a library for building fast, lightweight Web Components. At its core is a component base class that eliminates boilerplate code and provides responsive state, scoped styles, and a declarative templating system that are small, fast, and expressive.

Photoshop's UI elements come from Adobe's Web Components library: Spectrum[2], which implements Adobe's design system.

Spectrum Web Components have the following features:

Accessibility by default: Developed with existing and emerging browser specifications in mind to support assistive technologies.
Lightweight: Implemented using Lit Element with minimal overhead.
Standards-based: Built on Web Components standards such as Custom Elements and Shadow DOM.
Frame-agnostic: Can be used with any framework thanks to browser-level support.

Additionally, the entire Photoshop application is built using Lit-based Web Components. Lit's templates and virtual DOM differentiation make UI updates efficient. The encapsulated nature of Web Components also makes it possible to easily integrate React code from other teams when needed.

Overall, Web Components' browser-native custom elements combined with Lit's performance provide Adobe with the flexibility needed to build complex Photoshop UIs while maintaining efficiency.

Optimize Photoshop performance in browsers

Although new Web Components provide the foundation, intensive desktop applications like Photoshop still require extensive tracking and performance optimization work to provide a best-in-class online experience.

Use Service Workers to cache resources and code

Service Workers allow web applications to cache their code and resources locally after the user first accesses them, so that subsequent loads can be rendered faster. Although Photoshop doesn't yet support full offline use, it already takes advantage of Service Workers to cache its WebAssembly modules, scripts, and other resources to improve loading speeds.

Chrome DevTools Application panel > Cache storage shows the different types of resources that Photoshop pre-cache, including many JavaScript code blocks that are cached locally after code splitting on the web. These locally cached JavaScript code blocks make subsequent loading very fast. This caching mechanism has a huge impact on loading performance. After the first visit, subsequent loads are usually very fast.

Adobe uses the Workbox[3] library to more easily integrate Service Worker caching into the build process.

When resources are returned from the Service Worker cache, the V8 engine uses some optimization strategies:

Resources cached during installation are immediately compiled and code cached for consistent and fast performance.
Resources cached through the Cache API will undergo optimized caching processing when loaded for the second time, which is faster than ordinary caching.
V8 is able to perform more aggressive compilation optimizations based on the cache importance of resources.

These optimizations allow Photoshop's large cached WebAssembly modules to achieve higher performance.

Stream compilation and caching of large WebAssembly modules

Photoshop's code base requires multiple large WebAssembly modules, some of which are over 80MB in size. Streaming compilation in V8 and Chrome supports efficient processing of these large modules.

Additionally, when a WebAssembly module is first requested from a Service Worker, V8 generates and stores an optimized version for caching, which is critical for Photoshop's large code size.

Multithreading support for parallel graphics operations

In Photoshop, many core image processing operations, such as pixel transformations, can be significantly accelerated by executing them in parallel on multiple threads. WebAssembly's threading support enables taking advantage of multi-core devices for compute-intensive graphics tasks.

This allows Photoshop to port performance-critical image processing functions to WebAssembly and use the same multi-threading approach as on the desktop to achieve parallel processing.

Debugging optimization through WebAssembly

WebAssembly debugging support is important for diagnosing and resolving performance bottlenecks during development. Chrome DevTools has a series of functions such as analyzing WASM code, setting breakpoints and inspecting variables, which makes debugging of WASM have the same debuggability as JavaScript.

Integrating on-device machine learning with TensorFlow.js

Recent web versions of Photoshop include the ability to provide AI capabilities using TensorFlow.js[4]. Running models on-device rather than in the cloud improves privacy, latency, and cost-effectiveness.

TensorFlow.js is an open source machine learning library for JavaScript developers that can run in browser clients. It is the most mature option among Web machine learning solutions, supporting comprehensive WebGL and WebAssembly backend operators, and will also have the option of using the WebGPU backend in the future to achieve faster performance to adapt to new Web standards.

The Select Subject feature uses machine learning technology to automatically extract the main foreground objects in the image, greatly speeding up complex selections.

Below is an illustration of a sunset, and I want to change it to a night scene. Used "Select Topic" and AI prompts to try to select the most interesting areas for updates.

Photoshop can generate an updated illustration based on the AI prompt:

Following the AI prompt, Photoshop generated an updated illustration based on this:

The model has been converted from TensorFlow to TensorFlow.js to enable local execution:

// 加载选择主题模型

const model = wait tf.loadGraphModel('select_subject.json');



// 对图像张量运行推理

const {mask, background} = model.execute(imgTensor);



// 从掩码中细化选择

Adobe and Google collaborated to solve the synchronization problem between Photoshop's WebAssembly code and TensorFlow.js by developing a proxy API for Emscripten. This enables seamless integration between frameworks.

Since the Google team has improved the hardware execution performance of TensorFlow.js through its various supported backends (WebGL, WASM, Web GPU), this has improved the performance of the model by 30% to 200%, enabling near real-time in the browser performance.

Key models are optimized for performance-critical operations, such as Conv2D. Photoshop can choose to run models on the device or in the cloud, depending on performance needs.

The future of Photoshop on the Web

The ubiquity of Photoshop on the Web is a huge milestone, but it's just the tip of the iceberg of possibilities.

As browser vendors continue to develop and improve standards and capabilities, Photoshop will continue to expand on the Web, bringing more features online through progressive enhancement. And, Photoshop is just the beginning. Adobe plans to aggressively build its entire Creative Cloud suite on the web, unlocking more sophisticated design applications in the browser.

Adobe's collaboration with browser engineers will continue to advance the Web platform, raising standards and improving performance to develop more ambitious applications. What awaits us ahead is a future full of infinite possibilities!