foreword
Like everyone else, I am also thinking about how to use the existing Ai tools to effectively improve the activity of the app and diversify the services inside. Everyone has tried a lot on ChatGPT, so I plan to start with the Ai drawing tool.
But now Midjourney starts to charge, 30 knives a month is still quite expensive, so I gave up (after all, I am poor). So I remembered the Scribble Diffusion I used before. It only needs to draw two sketches to generate high-quality pictures. The experience is good and it is free.
Scribble Diffusion is a simple online service that uses AI to transform rough sketches into refined images, each one different (and free of copyright headaches). To put it simply, we only need to "draw a sketch with a brush, wait a moment after entering the description", and then a painting will be generated for you. This painting can be generated multiple times, and the result of each generation is also very different.
The ability of Scribble Diffusion is probably like this (the left is drawn by me, and the right is drawn by TA)
a photo of grassland with cloud
the sun setting behind the mountains
A lovely kitten
I found that the ability of Scribble Diffusion to draw is very unexpected, and you can define different photo styles (such as photos, oil paintings, sketches, etc.) (After all, it works out of the box and does not require any configuration. Friends who do not understand the technology of small program containers can pay attention to FinClip , mPaaS and other products).
After researching the official website , I found that the elements in the official website are very simple. Because of this, I think there are several steps to move "Scribble Diffusion" to the FinClip applet :
- Use canvas to realize the drawing board, and can draw in the applet;
- Provide an input box and a generated button, which can supplement the description of the picture and the generated button;
- Get the generated image link for display
Use applets to implement sketchpads
We can use a small program to implement a drawing board, and use the canvas tag to implement the brush function. Users can draw on the drawing board, or choose to clear the drawing board.
Here is a sample code:
<!--画布区域-->
<view class="canvas_area">
<canvas id="myCanvas" canvas-id="myCanvas" class="myCanvas"
disable-scroll="false"
bindtouchstart="touchStart"
bindtouchmove="touchMove"
bindtouchend="touchEnd">
</canvas>
</view>
<view class="clearBtn" bindtap="reset">
清空画板
</view>
复制代码Page({
data: {
isProcessing: false,
prompt: '',
scribble: null,
pen : 2, //画笔粗细默认值
color : '#000000', // 画笔颜色默认值
result: null,
text: ''
},
startX: 0, //保存X坐标轴变量
startY: 0, //保存X坐标轴变量
onLoad(params) {
wx.createSelectorQuery().select('#myCanvas').context((res) => {
this.context = res.context
}).exec()
},
//手指触摸动作开始
touchStart: function (e) {
//得到触摸点的坐标
this.startX = e.changedTouches[0].x
this.startY = e.changedTouches[0].y
// this.context = wx.createContext()
this.context.setStrokeStyle(this.data.color)
this.context.setLineWidth(this.data.pen)
this.context.setLineCap('round') // 让线条圆润
this.context.beginPath()
},
//手指触摸后移动
touchMove: function (e) {
var startX1 = e.changedTouches[0].x
var startY1 = e.changedTouches[0].y
this.context.moveTo(this.startX, this.startY)
this.context.lineTo(startX1, startY1)
this.context.stroke()
this.startX = startX1;
this.startY = startY1;
//只是一个记录方法调用的容器，用于生成记录绘制行为的actions数组。context跟<canvas/>不存在对应关系，一个context生成画布的绘制动作数组可以应用于多个<canvas/>
wx.drawCanvas({
canvasId: 'myCanvas',
reserve: true,
actions: this.context.getActions() // 获取绘图动作数组
})
},
//手指触摸动作结束
touchEnd: function () {
var imageData = wx.canvasGetImageData({
canvasId: 'myCanvas',
height: 250,
width: 250,
x: 0,
y: 0,
success(res){
return res.data
}
})
},
//清除画板
reset: function(){
this.context.clearRect(0, 0, 400, 400);
this.context.draw(true)
}
})
复制代码Provide input box and generate button
We need to provide an input input box for the user to enter the prompt; at the same time, we need to provide a button, which will trigger a response event when clicked, generate a picture from the canvas content, and submit the prompt input as a parameter to the server for picture generation.
Here is sample code:
<!-- 输入框 -->
<view class="imageDes">
<view class="formInput">
<input class="input" type="text" name="go" placeholder="用关键词描述画的内容" bindinput="update"/>
</view>
</view>
复制代码Page({
... 省略上述代码
// 更新表单提交按钮状态
update(e){
this.setData({
prompt : e.detail.value
})
},
})
复制代码Get the generated image link and display it
When the user clicks the Generate Image button, we will submit the canvas content and the prompt entered by the user as parameters to the server for image generation. The server will return the generated image link, and we need to display it to the user.
In the sample code below, our server sends a POST request, then parses the returned JSON data, gets the image link, and adds it to the page. The user can see the generated picture.
<!-- 绘图结果 -->
<view class="result" wx:if="{{result}}">
<view class="resultBox">
<view class="content">
<image class="content" src="{{result}}" mode="aspectFit" />
</view>
<view class="download">
<view class="btn" bindtap="download">
下载
</view>
</view>
</view>
</view>
复制代码Page({
... 省略上述代码
async getCanvasImage() {
return new Promise((resolve, reject) => {
wx.canvasToTempFilePath({
x: 0,
y: 0,
width: 250,
height: 250,
destWidth: 250,
destHeight: 250,
canvasId: 'myCanvas',
success(res) {
console.log(res.tempFilePath)
resolve(res.tempFilePath)
},
fail(err) {
console.log(err)
}
})
})
},
async upload(image) {
return new Promise((resolve) => {
wx.uploadFile({
url: 'xxxxxx',
filePath: image,
name: 'file',
success (res){
const data = JSON.parse(res.data)
resolve(data.url)
},
fail(err){
console.log('上传失败')
console.log(err)
}
})
})
},
async sleep(time) {
return new Promise((resolve) => {
setTimeout(() => {
resolve()
}, time)
})
},
async handleSubmit(e) {
if (!this.data.prompt) {
return
}
wx.showLoading({
title: '生成中',
})
try {
const prompt = this.data.prompt
const image = await this.getCanvasImage()
this.setData({
error: null,
isProcessing: true
});
const url = await this.upload(image)
console.log('图片', url)
const body = {
prompt: prompt,
image: url,
};
const response = await my_fetch.fetch( {
url: "https://scribblediffusion.com/api/predictions",
method: "POST",
headers: {
"Content-Type": "application/json",
},
params: JSON.stringify(body),
});
let prediction = response.data;
console.log('预测', prediction)
if (response.statusCode !== 201) {
wx.showToast({
title: '生成失败',
duration: 2000
})
this.setData({
error: '生成失败'
});
return;
}
while (
prediction.status !== "succeeded" &&
prediction.status !== "failed"
) {
console.log(prediction.status)
await this.sleep(500);
const response = await my_fetch.fetch({
url:"https://scribblediffusion.com/api/predictions/" + prediction.id,
});
prediction = response.data;
if (response.statusCode !== 200) {
this.setData({
error: prediction.detail
});
return;
}
}
if (Array.isArray(prediction.output) && prediction.output.length > 1) {
wx.hideLoading()
this.setData({
isProcessing: false,
result: prediction.output[1]
});
} else {
wx.hideLoading()
wx.showToast({
title: '生成失败',
duration: 2000
})
this.setData({
isProcessing: false,
error: '生成失败'
})
}
} catch (error) {
wx.hideLoading()
console.log(error)
wx.showToast({
title: '生成失败',
duration: 2000
})
}
},
})
复制代码After generating the applet, upload the applet through the management background to get the ai drawing tool in the App!
Of course, if you have more strange ideas about Scribble Diffusion and want to put them into practice, the developer has also open sourced the project files, so you can try them out~