CameraX-Beta01: ImageCapture Use Case

# Plan

After the preview, we can capture a frame into memory as ImageView; and we can save it into the shared media storage. In this article, let me show you how the CameraX saves the image in both pre-Android-Q and post-Android-Q version. The plan is "Design UI", "Setup Use Case", and "Set Actions".

# Build.Gradle: Update to Beta01

// Use the most recent version of CameraX, currently that is alpha10.
def camerax_version = '1.0.0-beta01'
// CameraX core library using the camera2 implementation
//implementation "androidx.camera:camera-core:${camerax_version}"
implementation "androidx.camera:camera-camera2:${camerax_version}"
// If you want to use the CameraX View class
implementation "androidx.camera:camera-view:1.0.0-alpha08"
// If you want to use the CameraX Extensions library
implementation "androidx.camera:camera-extensions:1.0.0-alpha08"
// If you want to use the CameraX Lifecycle library
implementation "androidx.camera:camera-lifecycle:${camerax_version}"

# UI: ImageView to display the captured image

I add one ImageView, ImageButton and panel layout.

<ImageView
    android:id="@+id/captured_ImageView"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:layout_gravity="center"
    android:layout_marginStart="32dp"
    android:layout_marginTop="32dp"
    android:layout_marginEnd="32dp"
    android:layout_marginBottom="32dp"
    android:contentDescription="@string/imageOverlay"
    android:visibility="gone"
    app:layout_constraintBottom_toBottomOf="parent"
    app:layout_constraintEnd_toEndOf="parent"
    app:layout_constraintStart_toStartOf="parent"
    app:layout_constraintTop_toTopOf="parent" />

<ImageButton
    android:id="@+id/capture_button"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:layout_marginBottom="@dimen/margin_small"
    android:contentDescription="@string/capture_image"
    app:layout_constraintBottom_toBottomOf="parent"
    app:layout_constraintEnd_toEndOf="parent"
    app:layout_constraintStart_toStartOf="parent"
    app:srcCompat="@drawable/ic_camera" />

<androidx.constraintlayout.widget.ConstraintLayout
    android:id="@+id/panel_ConstraintLayout"
    android:layout_width="0dp"
    android:layout_height="wrap_content"
    android:layout_marginBottom="@dimen/button_margin_bottom"
    android:foregroundGravity="center_horizontal"
    android:orientation="horizontal"
    android:visibility="gone"
    android:weightSum="2"
    app:layout_constraintBottom_toBottomOf="parent"
    app:layout_constraintEnd_toEndOf="parent"
    app:layout_constraintStart_toStartOf="parent">

    <LinearLayout
        android:id="@+id/panel_LinearLayout"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:orientation="horizontal"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent">

        <com.google.android.material.floatingactionbutton.FloatingActionButton
            android:id="@+id/reject_FloatingButton"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:clickable="true"
            android:focusable="true"
            android:src="@drawable/ic_close"
            app:backgroundTint="@color/rejectedRed"
            app:srcCompat="@drawable/ic_close" />

        <com.google.android.material.floatingactionbutton.FloatingActionButton
            android:id="@+id/accept_FloatingButton"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_marginStart="150dp"
            android:clickable="true"
            android:focusable="true"
            android:src="@drawable/ic_check"
            app:backgroundTint="@color/acceptedGreen"
            app:srcCompat="@drawable/ic_check" />
    </LinearLayout>

</androidx.constraintlayout.widget.ConstraintLayout>

Please be noted: The visibilities of ImageView and panel layout are set as GONE before the user activates them.

# Buttons

On the screen, you need a button to capture the image. Also, I add an extra panel to deal with image saving process. Here is code for the buttons:

// image capture
private lateinit var mBitmap: Bitmap

// view
private val overlayImage by lazy { 
findViewById<ImageView>(R.id.captured_ImageView) }

private val savePanel by lazy { 
findViewById<ConstraintLayout>(R.id.panel_ConstraintLayout) }

// button
private val captureButton by lazy { 
findViewById<ImageButton>(R.id.capture_button) }

private val acceptButton by lazy { 
findViewById<FloatingActionButton>(R.id.accept_FloatingButton) }

private val rejectButton by lazy { 
findViewById<FloatingActionButton>(R.id.reject_FloatingButton) }


private fun setButtons() {
    captureButton.setOnClickListener {
        ...
    }

    acceptButton.setOnClickListener {
        ...
    }

    rejectButton.setOnClickListener {
        ...
    }
}

Reader: What are the "..."? Homan: You need some actions. Come on! Fill in the blank by yourself.

* captureButton: Take captured frame to the capturedImageView and display the panelConstraintLayout.

* acceptButton: Perform the saveImage function and hide panelConstraintLayout.

* rejectButton: cancel the action and hide panelConstraintLayout.

I deal with some actions later. You can leave them blank.

# Setup ImageCapture Use Case

We need to set the options for ImageCapture.

// Bind the CameraProvider to the LifeCycleOwner
cameraProviderFuture.addListener(Runnable {
    // CameraProvider
    cameraProvider = cameraProviderFuture.get()

    // Preview
    preview = setPreview()

    // Capture
    imageCapture = setImageCapture()

I put it into a function.

// Use case: ImageCapture
private fun setImageCapture(): ImageCapture {
    // Build the image capture use case and attach button click listener
    val imageCapture = ImageCapture.Builder().apply {
        setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY)
        setTargetAspectRatio(screenAspectRatio)
        setTargetRotation(rotation)
    }.build()

    return imageCapture
}

I set the capture speed, image format matched with screen and set the rotation. Next, let's bind it into the lifecycle.

// Capture
imageCapture = setImageCapture()

// Must unbind the use-cases before rebinding them.
cameraProvider.unbindAll()

try {
    cameraStarted = true

    // A variable number of use-cases can be passed here -
    // camera provides access to CameraControl & CameraInfo
    camera = cameraProvider.bindToLifecycle(
        this as LifecycleOwner, cameraSelector, preview, imageCapture)

} catch(exc: Exception) {
    lge("Use case binding failed: $exc")
}

Here is the screen aspect ratio:

private var screenAspectRatio: Int = 0
fun setScreenRatio() {
    // Get screen metrics used to setup camera for full screen resolution
    val metrics = DisplayMetrics().also { viewFinder.display.getRealMetrics(it) }
    lgd("Screen metrics: ${metrics.widthPixels} x ${metrics.heightPixels}")

    screenAspectRatio = aspectRatio(metrics.widthPixels, metrics.heightPixels)
}

private fun aspectRatio(width: Int, height: Int): Int {
    val previewRatio = max(width, height).toDouble() / min(width, height)
    if (abs(previewRatio - RATIO_4_3_VALUE) <= abs(previewRatio - RATIO_16_9_VALUE)) {
        return RATIO_4_3
    }
    return RATIO_16_9
}

# Action of Capture Button

Now, we can input the action of the capture button.

captureButton.setOnClickListener {
    // get start time
    mStartCaptureTime = SystemClock.elapsedRealtime();

    // image capture
    imageCapture.takePicture(executor,
        object : ImageCapture.OnImageCapturedCallback() {
            override fun onCaptureSuccess(
                image: ImageProxy
            ) {
                lgd("Image format: ${image.format}")

                // show captured image
                mBitmap = imageProxyToBitmap(image)

                image.close()

                // show panel
                showSavePanel(true)
            }

            override fun onError(exception: ImageCaptureException) {
                super.onError(exception)
            }
        })
}

In the listener, we use OnImageCapturedCallback() to capture the frame into the memory. And then, we can decide how to use the data in our app. In CameraX demo from Google Codelab, it skips this callback to save the file. So this callback is optional. In this callback, the imageProxyToBitmap transfers data to the Bitmap so we can see it on screen.

// Convert data to Bitmap
private fun imageProxyToBitmap(image: ImageProxy): Bitmap {
    val planeProxy: ImageProxy.PlaneProxy = image.planes[0]

    val buffer: ByteBuffer = planeProxy.buffer
    val bytes = ByteArray(buffer.remaining())
    buffer.get(bytes)

    return BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
}

Here is showSavePanel(Boolean).

// Show Save panel to user
private fun showSavePanel(askForSave: Boolean) {
    if (askForSave) {
        lgd("Show save panel...")
        // stop preview session
        runOnUiThread {
            cameraProvider.unbind(preview)
            // view
            lgi("Show captured image at ImageView.")
            savePanel.visibility = View.VISIBLE
            captureButton.visibility = View.GONE
            viewFinder.visibility = View.GONE

            // display ImageView
            alignWithScreen()
            overlayImage.setImageBitmap(mBitmap)

            overlayImage.visibility = View.VISIBLE
            overlayImage.invalidate()
        }
    } else {
        lgd("Hide save panel...")
        // view
        runOnUiThread {
            overlayImage.visibility = View.GONE
            savePanel.visibility = View.GONE
            captureButton.visibility = View.VISIBLE
            viewFinder.visibility = View.VISIBLE
        }
        // restart camera
        viewFinder.post { startCamera() }
    }
}

Sometimes, the phone shows the bitmap wrong, such as this image.

So you need to align the bitmap to the screen.

// fix the bitmap which is not align with screen
private fun alignWithScreen() {
    val metrics = DisplayMetrics().also { viewFinder.display.getRealMetrics(it) }
    lge("screen: w = ${metrics.widthPixels}; h = ${metrics.heightPixels}")
    lgd("bitmap: w = ${mBitmap.width}; h = ${mBitmap.height}")
    if ((metrics.widthPixels > metrics.heightPixels) != (mBitmap.width > mBitmap.height)) {
        mBitmap = BitmapRotate(mBitmap, 90f)
        lgi("fixed bitmap: w = ${mBitmap.width}; h = ${mBitmap.height}")
    }
}

I rotate the bitmap in 90 degrees.

fun BitmapRotate(source: Bitmap, degrees: Float): Bitmap {
    val matrix = Matrix().apply { postRotate(degrees) }
    return Bitmap.createBitmap(source, 0, 0, source.width, source.height, matrix, true)
}

Logcat:

D/MYLOG MainActivity: Show save panel...

I/MYLOG MainActivity: Show captured image at ImageView.

E/MYLOG MainActivity: screen: w = 1080; h = 2160

D/MYLOG MainActivity: bitmap: w = 1280; h = 720

I/MYLOG MainActivity: fixed bitmap: w = 720; h = 1280

Now, we are on the right path.

# Action of Accept Button: saveImage()

acceptButton.setOnClickListener {
    lgi("accept button clicked!")
    saveImage()
}

Now, we reach the main part of this exercise. I need to separate codes to support pre-Q and post-Q. Let's QQ.

// save captured image
private fun saveImage() {

    imageName = getFileName()

    // Create options
    var outputFileOptions: ImageCapture.OutputFileOptions? = null

    if (SDK_INT < Build.VERSION_CODES.Q) {

Also, I need to define some strings in the companion object.

private const val GALLERY_TITLE = "face_check_in"
private const val FILENAME_FORMAT = "yyyy-MM-dd-HH-mm-ss-SSS"
private const val PHOTO_EXTENSION = ".jpg"
private const val AUTHORITIES = "com.homan.huang.cameraxfacecheckin.fileprovider"
private const val EXTERNAL_IMG_PATH = "/storage/emulated/0/Pictures"
private const val RATIO_4_3_VALUE = 4.0 / 3.0
private const val RATIO_16_9_VALUE = 16.0 / 9.0
private const val MINE_TYPE_JPEG= "image/jpg"
private const val DEFAULT_NAME = "easter_egg"

Define a filename.

private fun getFileName(): String {
    return SimpleDateFormat(FILENAME_FORMAT, Locale.US)
        .format(System.currentTimeMillis()) + PHOTO_EXTENSION
}

* Pre-Android-Q

 // image path
val imageDir: File = getOutputDirectory(this, GALLERY_TITLE)
lgd("image dir: ${imageDir.absolutePath}")

// path condition
val isImageDirCeated: Boolean = imageDir.exists()

if (isImageDirCeated) {
    // ImageIO, SDK < Android Q
    val newImage = createFile(imageDir, imageName)
    lgd("new image: ${newImage.absolutePath}")
    outputFileOptions = ImageCapture.OutputFileOptions.Builder(newImage).build()
} else {
    val msg = "Failed to create image directory!"
    shortMsg(this, msg)
    lge(msg)
}

I need to define the directory in the companion object.

/** Use external media if it is available, our app's file directory otherwise */
@Throws(IOException::class)
fun getOutputDirectory(context: Context, gallery: String): File {
    val path = File(EXTERNAL_IMG_PATH)
    lge("external path: $path")

    val newFolder = File(path, gallery)
    checkFolder(newFolder)
    lgd("new folder: ${newFolder.path}")

    return newFolder
}

Check folder existence.

private fun checkFolder(f: File) { if (!f.exists()) f.mkdir()  }

After that, we can save the image by creating a blank file.

/** Helper function used to create a timestamped file */
private fun createFile(baseFolder: File, fileName: String): File {
    return File(baseFolder, fileName)
}

That's a lot of work. Let's see the post-Android-Q version.

* Post-Android-Q

// use content value to process the image
val imageValue = ContentValues().apply{
    put(MediaStore.MediaColumns.DISPLAY_NAME, imageName)
    put(MediaStore.MediaColumns.MIME_TYPE, MINE_TYPE_JPEG)
    put(MediaStore.MediaColumns.RELATIVE_PATH, "Pictures/$GALLERY_TITLE")
}

outputFileOptions = ImageCapture.OutputFileOptions.Builder(
    contentResolver,
    EXTERNAL_CONTENT_URI,
    imageValue).build();

That's it, so short. You need to define a relative path in ContentValue. Everything will be done for you automatically.

# OutputFileOption

Let's use OutputFileOption for OnImageSavedCallback{}.

if (outputFileOptions != null) {
    imageCapture.takePicture(
        outputFileOptions,
        getMainExecutor(this),
        object : ImageCapture.OnImageSavedCallback {

            override fun onImageSaved(outputFileResults: ImageCapture.OutputFileResults) {

                lgd("Saved image to ${outputFileResults}")

                val duration = SystemClock.elapsedRealtime() - mStartCaptureTime

                runOnUiThread {
                    shortMsg(this@MainActivity, "Image captured in $duration ms")
                    showSavePanel(false)
                }
            }

            override fun onError(exception: ImageCaptureException) {
                lge("Failed to save image ${ exception.cause }")
            }
        })
} else {
    val msg = "Failed to capture image!"
    shortMsg(this, msg)
    lge(msg)
}

That's your only option in Beta-01 to save images by CameraX. Let's test the function on phones. If you don't have an Android-Q phone, you can try on an emulator.

Here is the test result from Android-Q emulator.

New gallery created: Pass

New image created: Pass