Android Camera2 learning notes

Directly speaking of Camera2, please refer to the article (this article is a learning note of the following article, and it is recommended to read the following article directly)

https://www.jianshu.com/p/9a2e66916fcb Overview

https://www.jianshu.com/p/df3c8683bb90 Switch

https://www.jianshu.com/p/067889611ae7 Preview

https://www.jianshu.com/p/2ae0a737c686 Take pictures

overview

1. Process of taking photos

As shown in the figure below, the API model of Camera2 is designed as a Pipeline, which processes the requests of each frame in sequence and returns the request results to the client.

2. Capture

Capture is more than just taking pictures. In fact, all operations in Camera2 are abstracted as capture, such as focusing.

Capture can be subdivided into single mode, multiple mode and repeated mode

  • One shot: refers to the Capture operation that is performed only once, such as setting flash mode, focusing mode and taking a picture. The Capture in multiple one shot modes will enter the queue and be executed in sequence
  • Burst mode: it refers to performing the specified Capture operation for multiple times in succession. Note that it is not allowed to insert any other Capture operation during multiple Capture execution, such as taking 100 photos in succession
  • Repeating: refers to the repeated execution of the specified Capture operation. When a Capture in other modes is submitted, the mode will be suspended and the Capture in other modes will be executed instead. When the Capture in other modes is executed, the execution will be resumed. Such as preview, etc

3. CameraManager

CameraManager has few functions. It is mainly used to query camera information and open the camera

4. CameraCharacteristics

Camera characteristics carries a lot of camera information, such as camera orientation, judging whether the flash is available, AE mode, etc

5. CameraDevice

Indicates the currently connected camera. Through CameraDevice, the following operations can be performed

  • Create CameraCaptureSession
  • Create CaptureRequest
  • Turn off camera operation (open through CameraManager)
  • Monitor device status

6. Surface

Surface is a memory space used to fill the image. For example, you can use the surface of SurfaceView to receive each frame of preview data for displaying the preview screen, or you can use ImageReader and surface to receive JPEG or YUV data. Each surface can have its own size and data format. You can obtain the list of sizes supported by a data format from CameraCharacteristics

7. CameraCaptureSession

As the name suggests, CameraCaptureSession is a "session" established with the camera, and the subsequent CaptureRequest will submit the reply

8. CameraRequest

A camera operation request, such as taking photos, previewing, etc

Switch camera

1. Camera configuration

Add the following code to Android manifest

    <uses-permission android:name="android.permission.CAMERA" />
    <uses-permission android:name="android.permission.INTERNET" />

    <uses-feature android:name="android.hardware.camera" />

Dynamically apply for camera permission (too simple, omitted)

2. Get camera information

    private val mCameraManager by lazy {
        mContext.getSystemService(CameraManager::class.java) as CameraManager
    }

    /**
     * Get CameraId after initialization
     */
    fun initCameraId() {
        if (mBackCameraId.isNotEmpty() || mFortCameraId.isNotEmpty()) return
        // mCameraManager is an instance of CameraManager
        for (cameraId in mCameraManager.cameraIdList) {
            val cameraCharacter = mCameraManager.getCameraCharacteristics(cameraId)
            val facing = cameraCharacter.get(CameraCharacteristics.LENS_FACING)
            //The camera faces. It's very simple here. We can get the first camera respectively
            if (facing == CameraCharacteristics.LENS_FACING_FRONT) {
                if (mFortCameraId.isEmpty()) mFortCameraId = cameraId
            } else if (facing == CameraCharacteristics.LENS_FACING_BACK) {
                if (mBackCameraId.isEmpty()) {
                    mBackCameraId = cameraId
                }
            }
        }
    }

2. Turn on the camera

The code is as follows

    fun openCamera(surface: Surface) {
        if (isOpenCamera()) return
        initCameraId()
        //Check permissions
        if (ActivityCompat.checkSelfPermission(mContext, Manifest.permission.CAMERA) != PackageManager.PERMISSION_GRANTED) {
            return
        }
        //Turn on the camera
        mCameraManager.openCamera(mBackCameraId, mCameraStateCallback, mBackGroundHandler)
        mPreviewSurface = surface
    }


    private val mCameraStateCallback: CameraDevice.StateCallback = object : CameraDevice.StateCallback() {
        override fun onOpened(camera: CameraDevice) {
            logcat("camera onOpen")
            // Get the CameraDevice instance, which is needed for subsequent operations
            mCameraDevice = camera
            openPreviewSession()
        }

        override fun onDisconnected(camera: CameraDevice) {
            logcat("camera on disconnect")
            camera.close()
            mCameraDevice = null
        }

        override fun onClosed(camera: CameraDevice) {
            logcat("camera onClosed")
            mCameraDevice = null
        }

        override fun onError(camera: CameraDevice, error: Int) {
            logcat("camera onError")
            camera.close()
            mCameraDevice = null
        }
    }

    private val mBackGroundHandler by lazy {
        Handler(mBackGroundHandlerThread.looper)
    }

    private val mBackGroundHandlerThread by lazy {
        val thread = HandlerThread("back ground")
        thread.start()
        thread
    }

3. Turn off the camera

    fun closeCamera() {
        mCameraDevice?.close()
        mCameraDevice = null
    }

preview

1. Get Surface

We use surface texture to display the preview screen

        <TextureView
            android:id="@+id/camera_preview"
            android:layout_width="0dp"
            android:layout_height="0dp"
            app:layout_constraintBottom_toBottomOf="parent"
            app:layout_constraintLeft_toLeftOf="parent"
            app:layout_constraintRight_toRightOf="parent"
            app:layout_constraintTop_toTopOf="parent" />

We set the SurfaceTextureListener and create the Surface after success

        binding.cameraPreview.surfaceTextureListener = object : TextureView.SurfaceTextureListener {
            override fun onSurfaceTextureAvailable(surface: SurfaceTexture?, width: Int, height: Int) {
                if (surface == null) return
                //Get the preview size. It's not important for the main process. It's written according to the mood
                val previewSize = MyCameraManager.instance.getPreviewSize(binding.cameraPreview.height, binding.cameraPreview.width) ?: return
                surface.setDefaultBufferSize(previewSize.width, previewSize.height)
                val lp = binding.cameraPreview.layoutParams
                lp.width = previewSize.height
                lp.height = previewSize.width
                // Sets the preview control size according to the preview size
                binding.cameraPreview.layoutParams = lp
                //Get previewed Surface
                mPreviewSurface = Surface(surface)
                if (!MyCameraManager.instance.isOpenCamera() && mIsResumed) {
                    //Turn on the camera
                    MyCameraManager.instance.openCamera(mPreviewSurface!!)
                }
            }

            override fun onSurfaceTextureSizeChanged(surface: SurfaceTexture?, width: Int, height: Int) {
                logcat("onSurfaceTextureSizeChanged")
            }

            override fun onSurfaceTextureDestroyed(surface: SurfaceTexture?): Boolean {
                logcat("onSurfaceTextureDestroyed")
                return true
            }

            override fun onSurfaceTextureUpdated(surface: SurfaceTexture?) {
//                logcat("onSurfaceTextureUpdated")
            }
        }

2. Preview

On base note, we have shown the operation of opening the camera, and then call openPreviewSession to create a session after opening the camera. The specific implementation code is as follows

    private fun openPreviewSession() {
        val surface = mPreviewSurface ?: return
        val device = mCameraDevice ?: return
        val characteristics = mCameraManager.getCameraCharacteristics(mBackCameraId)
        //Get the size of the preview, which is not important for this tutorial. Here it is based on your own mood
        val imageSize = getOptimalSize(characteristics, ImageReader::class.java, maxWidth = 1920, maxHeight = 1080)
        if (imageSize != null) {
            logcat("ImageSize:${imageSize.width} ${imageSize.height}")
            val jpegReader = ImageReader.newInstance(imageSize.width, imageSize.height, ImageFormat.JPEG, 2)
            jpegReader.setOnImageAvailableListener(OnJpegImageAvailableListener(), mBackGroundHandler)
            mJpegSurface = jpegReader.surface
            //Two surfaces are added here, one is the previewed Surface, and the other is the photographed Surface
            device.createCaptureSession(listOf(surface, mJpegSurface), mSessionStateCallback, mBackGroundHandler)
        }
    }

    private var mSessionStateCallback: CameraCaptureSession.StateCallback = object : CameraCaptureSession.StateCallback() {
        override fun onConfigured(session: CameraCaptureSession) {
            logcat("session onConfigured")
            val camera = mCameraDevice ?: return
            val surface = mPreviewSurface ?: return
            mSession = session

            //When the Session is created successfully, a request is issued
            val requestBuilder = camera.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW)
            //Here is our preview of the Surface
            requestBuilder.addTarget(surface)
            val request = requestBuilder.build()
            // Initiate a Repeating Request
            session.setRepeatingRequest(request, object : CameraCaptureSession.CaptureCallback() {
                override fun onCaptureStarted(session: CameraCaptureSession, request: CaptureRequest, timestamp: Long, frameNumber: Long) {
                }

                override fun onCaptureProgressed(session: CameraCaptureSession, request: CaptureRequest, partialResult: CaptureResult) {
                }

                override fun onCaptureCompleted(session: CameraCaptureSession, request: CaptureRequest, result: TotalCaptureResult) {
                }
            }, mBackGroundHandler)
        }
    }

As above, we can display the preview image

photograph

Note: there may be compatibility problems in the part of taking photos. The following code shows that Xiaomi's mobile phone takes photos abnormally. Huawei is ok. There should still be some details that haven't been handled well.

Create a Surface for capturing photographic images

    private fun openPreviewSession() {
        val surface = mPreviewSurface ?: return
        val device = mCameraDevice ?: return
        val characteristics = mCameraManager.getCameraCharacteristics(mBackCameraId)
        //Get the size of the preview, which is not important for this tutorial. Here it is based on your own mood
        val imageSize = getOptimalSize(characteristics, ImageReader::class.java, maxWidth = 1920, maxHeight = 1080)
        if (imageSize != null) {
            logcat("ImageSize:${imageSize.width} ${imageSize.height}")
            val jpegReader = ImageReader.newInstance(imageSize.width, imageSize.height, ImageFormat.JPEG, 2)
            jpegReader.setOnImageAvailableListener(OnJpegImageAvailableListener(), mBackGroundHandler)
            mJpegSurface = jpegReader.surface
            //Two surfaces are added here, one is the previewed Surface, and the other is the photographed Surface
            device.createCaptureSession(listOf(surface, mJpegSurface), mSessionStateCallback, mBackGroundHandler)
        }
    }

Here, you need to set ImageAvailableListener to save images

    private inner class OnJpegImageAvailableListener : ImageReader.OnImageAvailableListener {
        private val dateFormat: DateFormat = SimpleDateFormat("yyyyMMddHHmmssSSS", Locale.CHINA)
        private val cameraDir: String = "${Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DCIM)}/Camera"

        override fun onImageAvailable(reader: ImageReader?) {
            logcat("onImageAvailable:$reader")
            if (reader == null) return
            val image = reader.acquireNextImage()
            val captureResult = captureResults.take()
            if (image != null && captureResult != null) {
                logcat("image != null ")
                val jpegByteBuffer = image.planes[0].buffer
                val jpegByteArray = ByteArray(jpegByteBuffer.remaining())
                jpegByteBuffer.get(jpegByteArray)
                val width = image.width
                val height = image.height
                runIO {
                    val date = System.currentTimeMillis()
                    val title = "IMG_${dateFormat.format(date)}"
                    val displayName = "$title.jpeg"
                    val path = "$cameraDir/$displayName"
                    val orientation = captureResult[CaptureResult.JPEG_ORIENTATION]
                    val location = captureResult[CaptureResult.JPEG_GPS_LOCATION]
                    val longitude = location?.longitude ?: 0.0
                    val latition = location?.latitude ?: 0.0
                    logcat("write to local:$path")
                    File(path).writeBytes(jpegByteArray)

                    val values = ContentValues()
                    values.put(MediaStore.Images.ImageColumns.TITLE, title)
                    values.put(MediaStore.Images.ImageColumns.DISPLAY_NAME, displayName)
                    values.put(MediaStore.Images.ImageColumns.DATA, path)
                    values.put(MediaStore.Images.ImageColumns.DATE_TAKEN, date)
                    values.put(MediaStore.Images.ImageColumns.WIDTH, width)
                    values.put(MediaStore.Images.ImageColumns.HEIGHT, height)
                    values.put(MediaStore.Images.ImageColumns.ORIENTATION, orientation)
                    values.put(MediaStore.Images.ImageColumns.LONGITUDE, longitude)
                    values.put(MediaStore.Images.ImageColumns.LATITUDE, latition)

                    mContext.contentResolver.insert(MediaStore.Images.Media.EXTERNAL_CONTENT_URI, values)
                }
            }
        }
    }

Create a Request and initiate a photographing Request

    fun takePic() {
        val builder = mCameraDevice?.createCaptureRequest(CameraDevice.TEMPLATE_STILL_CAPTURE) ?: return
        builder.addTarget(mPreviewSurface!!)
        builder.addTarget(mJpegSurface!!)
        mSession?.capture(builder.build(), CaptureImageStateCallback(), mBackGroundHandler)
    }

Get CaptureRequest

    private inner class CaptureImageStateCallback : CameraCaptureSession.CaptureCallback() {
        override fun onCaptureCompleted(session: CameraCaptureSession, request: CaptureRequest, result: TotalCaptureResult) {
            logcat("onCaptureCompleted")
            super.onCaptureCompleted(session, request, result)
            captureResults.put(result)
        }
    }

 

That's it.

 

Tags: Android

Posted by joyser on Mon, 18 Apr 2022 04:21:17 +0930