How to Auto-Extract Text from Images in Firestore with Firebase Extensions

Introduction

With the rise of content-rich applications, the ability to extract text from images automatically can significantly enhance user experience. Firebase solves this need with the “Extract Image Text with Cloud Vision AI” extension. This extension uses Google Cloud’s Vision API to extract text from images stored in Cloud Storage and save it to Firestore documents, enabling a range of functionalities from content searching to automated metadata generation.

This guide will walk you through the setup and usage of the “Extract Image Text with Cloud Vision AI“ extension for your Firebase project.

Google Cloud Vision

Google Cloud Vision API allows developers to integrate vision detection features within applications, including image labeling, face and landmark detection, and Optical Character Recognition (OCR) for text extraction.

Benefits

• Versatile Text Detection: It can recognize and extract text from images of various formats.
• High Accuracy: Leveraging advanced machine learning models for text detection.
• Integrated Experience: Seamlessly connects with other Firebase services.
• Custom Configurations: Allows specifying storage paths for focused text extraction.

For more details, refer to the Google Cloud Vision documentation.

Getting Started with the Extract Image Text Extension

Before you begin, you’ll need a Firebase project with Cloud Firestore and Cloud Storage configured. Follow these steps to set up:

1. Create a Firebase project
2. Set up Cloud Storage within your project

Once your project is ready, add the Extract Image Text extension.

Extension Installation

You can install the “Extract Image Text with Cloud Vision AI” extension via the Firebase Console or Firebase CLI.

Option 1: Firebase Console

Visit the Firebase Extension’s page and locate the “Extract Image Text with Cloud Vision AI” extension. Click “Install in Firebase Console” and set up the configuration parameters.

Option 2: Firebase CLI

Alternatively, install the extension using the Firebase CLI with the command:

firebase ext:install googlecloud/storage-extract-image-text --project=your_project_id

Configure Extract Image Text Extension

Configuring the “Extract Image Text with Cloud Vision AI” extension is crucial to tailor its functionality to your specific requirements. Below are the configuration parameters with examples to guide you through the setup process.

Cloud Functions Location

Choose a location closest to your Firestore database to minimize latency.

Example:

• If your Firestore database is in Europe, you might choose europe-west1 as your Cloud Functions location.
• For a database in the US, us-central1 would be a suitable choice.
• Refer to the location selection guide.

Cloud Storage Bucket for Images

Specify the Cloud Storage bucket where you will upload images for text extraction.

Example: If your bucket’s name is devrel-extensions-testing.appspot.com, you will configure this parameter as devrel-extensions-testing.appspot.com.

Collection Path

Determine the Firestore collection path where the extracted text will be stored.

Example: If you want to store extracted text in a collection named extractedText, set this to extractedText.

Paths that Contain Images You Want to Extract Text From

You can restrict the extension to process images only from specified locations in your Storage bucket.

Example: To process images only from the essays and product-images directories, set this to /essays,/product-images.

List of Absolute Paths Not Included for Image Text Extraction

Define paths that should be excluded from text extraction. This is useful for ignoring specific directories.

Example: To exclude a temporary uploads folder and a private directory, set this parameter to /temporary-uploads,/private.

Amount of Text Extraction Information to Write to Firestore

Choose whether you want the basic extracted text or the full annotation data from the Cloud Vision API stored in Firestore.

Example:

• Select “basic” if you need just the extracted text for simplicity and storage efficiency.

• Choose “full” if you require detailed annotation data, such as bounding box coordinates and language hints, which can be helpful for advanced use cases.

With these configurations set up according to your specific needs, the “Extract Image Text with Cloud Vision AI” extension will be optimized for your Firebase project, ensuring efficient text extraction and storage tailored to your application’s requirements.

Text Extraction Workflow

After installation and configuration, the extension monitors the specified Cloud Storage bucket for new image uploads. Here’s the workflow:

1. Image Upload: When a JPG or PNG file is uploaded to the bucket
2. Extension: The extension triggers a Cloud Function.
3. Text Extraction: The Cloud Vision API processes the image, extracting any discernible text.
4. Firestore Document: The extracted text is then saved to the specified Firestore collection. In this collection, each document will have a field named file whose value matches the full storage path of the uploaded image.

Steps to Add Text Extraction to Firestore Documents

First, ensure your Firebase SDK is initialized and your app has Firestore and Cloud Storage operations permissions. Here’s an example of initializing Firebase in your web app:

const firebaseConfig = {
  // Your Firebase configuration keys
};

firebase.initializeApp(firebaseConfig);
const db = firebase.firestore();
const storage = firebase.storage();

Also, you already have a UI that can handle selecting image files, for example:

<!— HTML input element for file selection—>
<input type="file" id="image-upload" accept="image/*" />

JavaScript to handle the file upload:

document
  .getElementById('image-upload')
  .addEventListener('change', handleFileUpload);

function handleFileUpload(event) {
  // Get the file from the input element
  const file = event.target.files[0];
}

Step 1: Upload an Image to Cloud Storage

Upload an image to the specified Cloud Storage bucket. The extension will react to this event by extracting text from the image.

const imageRef = storage.ref('/essays/test-image.jpg');

imageRef.put(file).then((snapshot) => {
  console.log('Uploaded a blob or file!');
});

Step 2: Verify the Extracted Text in Firestore

After the upload, the extension processes the image and writes the extracted text to Firestore. You can verify this in the Firestore document.

const filePath = `gs://${imageRef.bucket}/${imageRef.name}`;

const textDocRef = db
  .collection('extractedTexts')
  .where('file', '==', filePath);

textDocRef.get().then((doc) => {
  if (doc.exists) {
    console.log('Extracted text:', doc.data().text);
  } else {
    console.log('No document found!');
  }
});

Step 3: Use the Extracted Text in Your Application

With the text now in Firestore, you can fetch and use it within your application.

Real-Time Updates and Display

To display text as soon as it’s extracted, set up a real-time listener on the Firestore document:

function listenForExtractedText(docId) {
  const docRef = db.collection('extractedTexts').doc(docId);

  docRef.onSnapshot((doc) => {
    if (doc.exists) {
      console.log('Updated text:', doc.data().text);
      // Update UI accordingly
    }
  });
}

Conclusion

The “Extract Image Text with Cloud Vision AI” Firebase Extension offers an automated and efficient way to extract and use text from images in Firestore. Following the steps outlined in this guide, you can enhance your application’s capabilities, making the most of Firebase’s robust ecosystem. Implement this extension to unlock new functionalities and offer users a more interactive experience.