1 Introduction
During this period, I am usingnatario1/CameraView to implement filtered 预览, a>, but the author has not fixed this problem. , there are many mentions of this in When taking photos with more than one filter, the preview will be normal, but the photos taken will be completely black. filters, it is normal to superimpose , superimposing Especially for taking pictures using has entered the deep water area, and it gradually appears that it cannot meet our needs. However, as the project continues to deepen, the use of the packaging is quite in place, it indeed saved us a lot of time in the early stages of the project. Since function. 拍照, 录像CameraViewCameraViewMultiFilter22
GithubissuesBUG
Previous article, we have reproduced this BUG,
and . In this article we will formally solve this problem the preview and photo effects are inconsistent. has explained whyAnother articleCameraViewBUG
2. Option 1
SinceCameraView the effects of previewing and taking pictures are inconsistent, the first thing that comes to mind is to re-implement the logic related to taking pictures by myself, so as to avoid eglSurface all kinds of problems. question.
2.1 Source code description
First of all, we know that the processing of filter photography is in the SnapshotGlPictureRecorder.take() method, and Snapshot2PictureRecorder is the implementation of Camera2. Is inherited from SnapshotGlPictureRecorder.
public class Snapshot2PictureRecorder extends SnapshotGlPictureRecorder {
//...省略了代码...
}
Snapshot2PictureRecorder is initialized inCamera2Engine’sonTakePictureSnapshot() method
@EngineThread
@Override
protected void onTakePictureSnapshot(@NonNull final PictureResult.Stub stub,
@NonNull final AspectRatio outputRatio,
boolean doMetering) {
stub.size = getUncroppedSnapshotSize(Reference.OUTPUT);
stub.rotation = getAngles().offset(Reference.VIEW, Reference.OUTPUT, Axis.ABSOLUTE);
mPictureRecorder = new Snapshot2PictureRecorder(stub, this,
(RendererCameraPreview) mPreview, outputRatio);
mPictureRecorder.take();
}
So if we want to replace the logic of taking photos with filters, we can just replace these two classes.
2.2 Replace SnapshotGlPictureRecorder
We create a new classMySnapshotGlPictureRecorder to replaceSnapshotGlPictureRecorder
@RequiresApi(Build.VERSION_CODES.LOLLIPOP)
public class MySnapshotGlPictureRecorder extends SnapshotPictureRecorder {
private RendererCameraPreview mPreview;
private AspectRatio mOutputRatio;
private Overlay mOverlay;
private boolean mHasOverlay;
public MySnapshotGlPictureRecorder(
@NonNull PictureResult.Stub stub,
@Nullable PictureResultListener listener,
@NonNull RendererCameraPreview preview,
@NonNull AspectRatio outputRatio,
@Nullable Overlay overlay) {
super(stub, listener);
mPreview = preview;
mOutputRatio = outputRatio;
mOverlay = overlay;
mHasOverlay = mOverlay != null && mOverlay.drawsOn(Overlay.Target.PICTURE_SNAPSHOT);
}
@Override
public void take() {
mPreview.addRendererFrameCallback(new RendererFrameCallback() {
@RendererThread
public void onRendererTextureCreated(int textureId) {
MySnapshotGlPictureRecorder.this.onRendererTextureCreated(textureId);
}
@RendererThread
@Override
public void onRendererFilterChanged(@NonNull Filter filter) {
MySnapshotGlPictureRecorder.this.onRendererFilterChanged(filter);
}
@RendererThread
@Override
public void onRendererFrame(@NonNull SurfaceTexture surfaceTexture,
int rotation, float scaleX, float scaleY) {
mPreview.removeRendererFrameCallback(this);
MySnapshotGlPictureRecorder.this.onRendererFrame(surfaceTexture,
rotation, scaleX, scaleY);
}
});
}
private void onRendererTextureCreated(int textureId) {
Rect crop = CropHelper.computeCrop(mResult.size, mOutputRatio);
mResult.size = new Size(crop.width(), crop.height());
/*if (CameraViewConstants.custom) {
// 如果自定义标志位生效，那么使用1920*1080，此处
mResult.size = new Size(1920, 1080);
}*/
}
private void onRendererFilterChanged(Filter filter) {
}
private void onRendererFrame(SurfaceTexture surfaceTexture, int rotation, float scaleX, float scaleY) {
//待实现...
}
}
HereonRendererTextureCreated, the size of the image is determined. Then in onRendererFrame, it is time to perform filter processing when taking pictures.
2.2.1 Get the size of the image
Advanced talentmResult.sizeCreatingwidthJapaneseheight
int width = mResult.size.getWidth();
int height = mResult.size.getHeight();
2.2.2 Reading image data
Then callGLES20.glReadPixels() to take out the pixel data that has been processed from the GPU frame buffer and save it in . Since it is in format, is transmitted as . OpenGLbufferRGBAcapacitywidth * height * 4
ByteBuffer buffer = ByteBuffer.allocateDirect(width * height * 4);
buffer.order(ByteOrder.nativeOrder());
GLES20.glReadPixels(
0,
0,
width,
height,
GLES20.GL_RGBA,
GLES20.GL_UNSIGNED_BYTE,
buffer
);
2.2.3 Create Bitamp
GeneralbufferTransformationBitmap
buffer.rewind();
Bitmap bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
bitmap.copyPixelsFromBuffer(buffer);
2.2.4 Using matrices for rotation processing
UsingMatrix matrix can mirror, rotate, etc. Bitmap. Here you can perform corresponding processing according to the actual camera hardware direction.
Matrix matrix = new Matrix();
matrix.preRotate(180,width/2F,height/2F);
Bitmap newBitmap = Bitmap.createBitmap(bitmap, 0, 0, width, height, matrix, false);
2.2.5 Convert Bitmap to Byte array
GeneralBitmapTrackingByteNumber combination
ByteArrayOutputStream stream = new ByteArrayOutputStream();
newBitmap.compress(Bitmap.CompressFormat.JPEG, 100, stream);
byte[] byteArray = stream.toByteArray();
try {
stream.close();
} catch (IOException e) {
throw new RuntimeException(e);
}
bitmap.recycle();
newBitmap.recycle();
2.2.6 Distribution data
assigns the Byte array to data, and calls dispatchResult for distribution, and finally distributes it to < callback list in a i=4>. CameraViewmListeners
mResult.data = byteArray;
dispatchResult();
We set the addCameraListener callback so that we can get the relevant data after calling takePictureSnapshot() to take a picture.
binding.cameraView.addCameraListener(object : CameraListener() {
override fun onPictureTaken(result: PictureResult) {
super.onPictureTaken(result)
//拍照回调
val bitmap = BitmapFactory.decodeByteArray(result.data, 0, result.data.size)
bitmap?.also {
Toast.makeText(this@Test2Activity, "拍照成功", Toast.LENGTH_SHORT).show()
//将Bitmap设置到ImageView上
binding.img.setImageBitmap(it)
val file = getNewImageFile()
//保存图片到指定目录
ImageUtils.save(it, file, Bitmap.CompressFormat.JPEG)
}
}
})
2.3 Create a new MySnapshot2PictureRecorder
NewMySnapshot2PictureRecorderInherited fromMySnapshotGlPictureRecorder
public class MySnapshot2PictureRecorder extends MySnapshotGlPictureRecorder {
public MySnapshot2PictureRecorder(@NonNull PictureResult.Stub stub,
@NonNull Camera2Engine engine,
@NonNull RendererCameraPreview preview,
@NonNull AspectRatio outputRatio) {
super(stub, engine, preview, outputRatio, engine.getOverlay());
}
@Override
public void take() {
super.take();
}
@Override
protected void dispatchResult() {
super.dispatchResult();
}
}
2.4 Replaced by MySnapshot2PictureRecorder
In of Camera2Engine, replace the initialization of with onTakePictureSnapshot()Snapshot2PictureRecorderMySnapshot2PictureRecorder
@EngineThread
@Override
protected void onTakePictureSnapshot(@NonNull final PictureResult.Stub stub,
@NonNull final AspectRatio outputRatio,
boolean doMetering) {
stub.size = getUncroppedSnapshotSize(Reference.OUTPUT);
stub.rotation = getAngles().offset(Reference.VIEW, Reference.OUTPUT, Axis.ABSOLUTE);
//Snapshot2PictureRecorder 替换为了MySnapshot2PictureRecorder
mPictureRecorder = new MySnapshot2PictureRecorder(stub, this,
(RendererCameraPreview) mPreview, outputRatio);
mPictureRecorder.take();
}
2.5 Run the program
Rerun the program and call the filter to take pictures, you can find thatCameraView叠加2个以上滤镜拍照黑屏的BUG has been solved.
Just when I was happy, I discovered that there was another problem: Every time after reassigning the fitler, the preview would flash black and then return to normal >
binding.cameraView.filter = multiFilter
This makes me very depressed. Is this too confusing? I didn’t have any ideas for a while. After sorting out the process many times, I still can only start from MultiFilter.
Fortunately, I don’t know whether it was a flash of inspiration or a coincidence that allowed me to find the second solution
3. Option 2
Let's roll back the code to the state before modifying plan 1, and then analyze the source code.
3.1 Source code analysis
MultiFilteronCreate()The method in We will create child elements in the draw() operation, because in Some may have been added after onCreate() was called. is empty, but there is such a line of comments:
@Override
public void onCreate(int programHandle) {
// We'll create children during the draw() op, since some of them
// might have been added after this onCreate() is called.
}
Let’s look at the draw() method again. Here we traverse the filter list filters and initialize each filters. Then draw again, and the next filter is drawn on the basis of the previous filter, thereby achieving the effect of filter superposition.
There is a problem, which means that every timedraw, it will be re-initializedfitler, then we can deduce, < The initialization of /span>. filter should originally be placed in onCreate()
@Override
public void draw(long timestampUs, @NonNull float[] transformMatrix) {
synchronized (lock) {
for (int i = 0; i < filters.size(); i++) {
boolean isFirst = i == 0;
boolean isLast = i == filters.size() - 1;
Filter filter = filters.get(i);
State state = states.get(filter);
maybeSetSize(filter);
maybeCreateProgram(filter, isFirst, isLast);
maybeCreateFramebuffer(filter, isFirst, isLast);
GLES20.glUseProgram(state.programHandle);
if (!isLast) {
state.outputFramebuffer.bind();
GLES20.glClearColor(0, 0, 0, 0);
} else {
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);
}
if (isFirst) {
filter.draw(timestampUs, transformMatrix);
} else {
filter.draw(timestampUs, Egloo.IDENTITY_MATRIX);
}
if (!isLast) {
state.outputTexture.bind();
} else {
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, 0);
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
}
GLES20.glUseProgram(0);
}
}
}
3.2 Solution implementation
general ondrawintermediatemaybeSetSize(), maybeCreateProgram(), maybeCreateFramebuffer()transition arrived a>onCreate()medium
public void onCreate(int programHandle) {
synchronized (lock) {
for (int i = 0; i < filters.size(); i++) {
boolean isFirst = i == 0;
boolean isLast = i == filters.size() - 1;
Filter filter = filters.get(i);
State state = states.get(filter);
maybeSetSize(filter);
maybeCreateProgram(filter, isFirst, isLast);
maybeCreateFramebuffer(filter, isFirst, isLast);
}
}
}
Other codes are still used
CameraViewThe original code, that is, all the codes modified in plan 1 are rolled back, and they are still usedSnapshotGlPictureRecorderandSnapshot2PictureRecorder
3.3. Run the program
Rerun the program and you can find that the two bugs CameraView叠加2个以上滤镜拍照黑屏的BUG and 每次重新赋值fitler之后，预览都会黑屏闪一下，才回复正常 have been resolved!
4. Others
4.1 CameraView source code analysis series
Android Camera Library CameraView Source Code Analysis (1): Preview-CSDN Blog
Android Camera Library CameraView Source Code Analysis (2): Taking Photos-CSDN Blog
Android camera library CameraView source code analysis (3): Filter related class description-CSDN Blog
Android camera library CameraView source code analysis (4): Taking photos with filters-CSDN Blog< a i=4>Android camera library CameraView source code analysis (5): Saving filter effects-CSDN Blog
4.2 Solve CameraViewBUG
Android solves the BUG of a black screen when taking pictures with CameraView superimposed with more than 2 filters (1): Recurring BUG-CSDN Blog
Android solves the BUG of a black screen when taking pictures with CameraView superimposed with more than 2 filters (2): Solving the BUG-CSDN Blog
Why are the preview and photo effects of the camera library CameraView inconsistent?-CSDN Blog