A low-code platform based on remote modules

This article is synchronized on the personal blog shymean.com , welcome to pay attention

In the previous article on thinking about low-code platforms , I proposed an idea of ​​using "dynamic rendering of asynchronous components" to avoid the problem of poor scalability of componentized low-code platforms. Based on this idea, I also studied the online preview of Vue. The realization of components and the use of vite to load remote modules , finally realized a low-code platform that supports loading remote components, which is very interesting. This article mainly records the relevant ideas and implementation principles.

The entire project has been published on github , and a Docker composedevelopment environment is provided.

Some problems with local components

In a component-based low-code platform, the editor first needs to provide relevant components, and then the operator can build a page by dragging and dropping.

This solution is currently relatively process, but also encountered some problems in the process of use

Expansion components

When the existing components cannot meet the requirements, developers need to expand the component library. This expansion process is roughly divided into several steps such as development -> testing -> packaging and online -> user updating the editor, and the process is relatively long.

Limitations of parameter configuration

Another problem based on components is that some customized requirements, such as specific data sources, user judgment, etc., need to encapsulate relevant logic in components, and then expose some configuration items, which leads to the scalability of components is also very high Difference.

For example, if the click logic of a button component is configured, the supported click strategies can only be encapsulated into functions in advance, and then mapped into configurable enumeration values.

const clickHanler = {
    1: openPage,
    2: showMessage
    // ...
}
复制代码

When clickHanler can't meet the demand, it also needs to go deep into the component to iterate.

Some editors provide js code to be passed in, and then eval runs to expand configuration items, which requires users to have code capabilities, or leave the configuration to developers to implement.

Dynamic loading performance issues

For performance reasons, on the generated page, not all components of the editor are loaded, but in ()=>import(xxx)the form of dynamically loaded components.

动态加载的本质是通过JSONP的方式加载远程模块文件，对于某些复杂页面，可能会配置多个组件，比如20个，这导致需要发送20个js请求出去，可能会被浏览器同域名请求数量限制等影响页面

也许可以通过SSR解决这个问题。

编辑器

在使用vite加载远程模块这篇文章中我提到了远程组件的使用场景，其主要作用是将编译的的步骤延迟到用户访问页面之前再进行

• 远程组件，可以保持与本地编写代码基本一致的扩展性
• 延迟编译，可以减少了开发->打包->发布的时间

当然，编写远程组件的缺点也很明显，最大的问题是无法复原本地开发环境，后面会讨论一下。

基于远程模块的页面编辑器这个思路，我实现了一个vue-page-builder，下面演示一下大概的操作，同时介绍相关的实现原理。

流程演示

首先创建一个组件

然后进去编辑界面，左侧是组件源码，右侧是组件配置源码，可以开始编写代码了。

目前只实现了基本的代码编辑器，理想情况下是希望能达到与本地一样的开发体验。

组件代码编写并保存之后，就可以在编辑器的组件列表看见对应的组件了

也可以在控件配置区域传入相关的配置

看起来就跟这个组件在本地开发的一样，但这所有的过程都是在线完成的，也就是说，我们只要新建了一个组件，在编辑器那边就可以直接看见对应的组件，而无需再经过提交代码->打包->发布的流程。

基于远程组件的设定，开发人员就可以任意扩展远程组件列表，远程组件也是可以依赖远程组件的！

所有数据都是保存在一个后台服务器上面的，我们称之为【data server】

预览原理

整个流程是基于vite-plugin-remote-module实现的，该插件提供了一个虚拟模块loadRemoteComponent用于加载远程模块

借助这个接口，我们可以实现一个RemoteWidget组件，用来加载动态的远程模块

<script lang="ts">
// @ts-ignore
import {loadRemoteComponent} from '@vite-plugin-remote-module';

import {defineComponent, h, onMounted, shallowRef} from "vue";

export default defineComponent({
  name: "remoteWidget",
  props: {
    url: {
      type: String,
      default: ''
    },
    passedProps: {
      type: Object,
      default: () => {
        return {}
      },
    },
  },
  setup(props, context) {
    const compRef = shallowRef(null)

    onMounted(() => {
      if (!props.url) return
      loadRemoteComponent(props.url).then((comp: object) => {
        compRef.value = comp
      })
    })

    return () => {
      if (!compRef.value) return h('div', {class: 'remote-loading'}, ['远程模块加载中...'])
      return h(compRef.value, props.passedProps, context.slots)
    }
  }
})
</script>

<style scoped lang="scss">
.remote-loading {
  height: 100px;
  line-height: 100px;
  text-align: center;
}

</style>
复制代码

RemoteWidget组件有两个props

• url远程组件的地址
• passedProps透传给远程组件的props

vite-plugin-remote-module的本质是在vite的server.middlewares中添加了一个中间件，当请求到未被下载的远程模块时，会先将远程模块下载到本地，然后再返回本地模块内容。

基于这个限制，整个编辑器实际上是以vite dev启动的一个开发环境服务器，称之为【editor server】，这样才能实现可以随时通过vite server去加载新增的远程模块。

由于编辑器是内部系统，直接部署vite server到线上，直接使用理论上是可以的。可能需要考虑部署在线上多人访问编辑器时的性能问题。

但对于编辑器生成的页面，肯定不能再启动vite server来提供预览的功能，而是要提供生产环境的、可以被线上访问的编译后的资源。

延迟编译

编辑器生成的页面，得到的实际上是一堆序列化的JSON内容，用于描述当前页面的各种配置

对于用户端而言，需要通过id拿到当前页面的配置，然后渲染出对应的页面。

与编辑器预览区不同的是，这个页面是在生产环境上被用户访问的，因此我们需要在这里实现整个流程中”延迟编译“的部分

假定我们通过页面编辑器，得到的页面链接是http://localhost:9988/template?id=2，在用户访问这个页面后，需要

• 根据id找到页面配置
• 读取配置内容，加载远程模块，编译，得到最终的html、css、js代码，返回页面，通过缓存编译结果
• 下次访问时，如果缓存未失效，则直接返回返回结果

编译

所以肯定需要一个新的server来实现这些功能，我们称之为【compile server】

其代码类似于

router.get("/template", async (ctx: any) => {
  const id = ctx.request.query.id

  let html = readCache(id)
  if (!html) {
    // 写入内容，然后返回html
    const sfc = json2sfc(content)
    html = await build(id, sfc)

    writeCache(id, html, new Date().toString())
  }
  ctx.type = "html";
  ctx.body = html
});
复制代码

通过json2sfc将配置内容转换成SFC文件，然后再使用vite build打包，就可以得到静态资源了。

这里需要注意的是，rollup动态import是基于blob全量加载本地模块，因此在本地打包时，如果远程模块未被下载，就会报错。要想打生产包，我们需要提前将RemoteWidget里面动态加载的组件下载到本地。

由于我们知道页面的配置，也就知道了当前页面依赖的远程组件，我们可以将其先解析出来，然后作为静态依赖加入，这样就可以让vite-plugin-remote-module提前去下载各种依赖，最后再打包了

export default function json2sfc(vnode: string): string {
  const data: IRemoteWidget = JSON.parse(vnode)
  const remoteWidgetList = findRemoteWidgetList(data.children.filter(row => row.type === 'RemoteWidget'))

  // 提前加载需要的动态模块
  let importStr = ''
  let widgetMapStr = '{'
  remoteWidgetList.forEach((url: string, index: number) => {
    const name = `RemoteWidget${index}`
    importStr += `import ${name} from '@remote/${url}'\n`
    widgetMapStr += `'${url}':${name},`
  })
  widgetMapStr += '}'

  return `
<script>
import {h, defineComponent} from 'vue'
import AbstractContainer from '@/abstractContainer.vue'

${importStr}

const remoteWidgetMap = ${widgetMapStr}

export default defineComponent({
  name: "App",
  render(){
     const widget = ${vnode}
     return h(AbstractContainer,{widget,remoteWidgetMap})
  }
});
<\/script>

<style scoped lang="scss">

<\/style>`
}
复制代码

其中

• remoteWidgetMap的作用是将远程组件映射成已经被加载的本地组件，这样就不用再使用RemoteWidget来动态加载远程模块，也就避免了动态加载导致发送多个js资源请求的情况了。
• importStr静态加载远程模块，

在build方法中，通过buildVite调用vite build完成打包

import {build as buildVite} from 'vite'

export async function build(id: string, content: string) {
  const folder = `/temp/${id}`

  const entry = `${folder}/index.html`
  const app = `${folder}/App.vue`

  const buildTargetFile = resolve(entry)
  process.env.BUILD_TARGET_FILE = buildTargetFile

  await fs.ensureDir(resolve(folder))

  const entryHtml = getEntryHTMLTemplate(app)

  await fs.writeFile(buildTargetFile, entryHtml)
  await fs.writeFile(resolve(app), content)

  // todo SSR编译
  await buildVite(getInlineConfig())
  // todo 把编译后的结果上传到cdn
  const html = await fs.readFile(resolve(`/dist/${entry}`))
  return html.toString()
}
复制代码

这要求compile server需要提供所有远程组件都支持的开发环境，比如SCSS、TypeScript等功能。

最终就可以得到一个完整的html和bundle js资源，生产环境的问题也就解决了！

缓存编译结果

vite 默认编译包含数个远程组件的页面，大概需要花费数秒钟的时间，在响应返回前用户看到的是空白的页面。因此缓存就显得十分重要。

常规的做法是在编辑保存页面之后，就触发一次编译，然后缓存编译接口，将编译时机从用户首次访问提前到保存页面之后，这样就可以避免首个用户需要等待的情况了。

缓存是一把双刃剑，我们还需要保证组件和页面的编辑能得到最新的结果，因此需要一些策略

• 当页面编辑后，需要缓存失效
• 当页面依赖的远程组件改动后，需要缓存失效

dateList配置文件是从data server查到的相关文件的updatedAt，用来编辑

function readCache(id: string, dateList:string[]):string {
  const cache = cacheMap[id]
  if (!cache) return ''

  const {content, createdAt} = cache
  const isUpdated = dateList.find(date=>+new Date(createdAt) < +new Date(date))

  return isUpdated ? '' : content
}
复制代码

小结

在上面的整个流程中，我们提到了三个服务器

• data server，一个纯后端服务器，与数据库交互，负责提供数据操作和持久化相关的接口，也是远程组件和页面配置保存的地方
• editor server, the editor preview server, which is essentially one vite devServer, is responsible for the editor preview area can normally dynamically load remote components
• compile server, the compilation server, responsible for the final accessible page output by the editor

With vite-plugin-remote-modulethis, you can flexibly load remote modules in the preview area.

With pre-declaration, dynamically loaded remote modules can be replaced before vite build, which is equivalent to packaging a local project.

At present, the biggest problem of the whole project is that the development experience of online editing components cannot be compared with that of local development. This is also the problem that needs to be solved next. Maybe you can refer to Gitpod to browse the remote code locally through remote ssh and other methods. The developer only needs to confirm and save it finally.

This is also in line with my original vision for the low-code platform: for people without front-end capabilities, this editor can use a rich component library 0 code configuration page; after the user's front-end capabilities are enhanced, component libraries can be created and maintained, Still very convenient. All of this is based on vite, and vite is really easy to use!

The entire project has been published on github , and a Docker composedevelopment environment is provided. If you have any questions, please correct me.