Identifying vehicles at entry points, toll booths, or parking facilities using license plates is important for many businesses, however, manually reviewing the plates and typing out the information is slow and prone to human error. The Structured OCR task on the Abilities tab can act as a powerful Optical Character Recognition (OCR) tool, reading the text on a document and outputting it into a clean, structured format.

For example, with any standard license plate, the model should examine the image and categorize the data into specific fields. The model will pull the main alphanumeric sequence for the Plate Number and the state for the State Jurisdiction.

‍

{

         "content_type": "license_plate",

         "plate_number": “SAMPLE”,

         "jurisdiction": {

           "state_or_province": “California”

         }

}

In contrast, if a user uploads an image that is severely blurred, cut off, or covered by a harsh glare, the model will return:

‍

{

         "content_type": "no_plate_detected"

       }

.

Our expected inputs are images of identification documents, and the expected output will be a structured text format, a JSON file for this example, containing the extracted text from the target fields.

‍

SDK Tutorial

First, let’s define the ability:

from eyepop import EyePopSdk
from eyepop.data.data_types import InferRuntimeConfig, VlmAbilityGroupCreate, VlmAbilityCreate, TransformInto
from eyepop.worker.worker_types import CropForward, ForwardComponent, FullForward, InferenceComponent, Pop
import json




ability_prototypes = [
    VlmAbilityCreate(
        name=f"{NAMESPACE_PREFIX}.structured-OCR.read-license-plate",
        description="Extract the license plate",
        worker_release="qwen3-instruct",
        text_prompt=license_plate_prompt,
        transform_into=TransformInto(),
        config=InferRuntimeConfig(
            max_new_tokens=400,
            image_size=512
        ),
        is_public=False
    )
]

The prompt we can use here is:
‍
"You are given an image of a vehicle or a zoomed-in license plate. Print out the license plate without printing state symbol, replace with a space

If the image does not contain a visible license plate, return:..."

‍

Next, we can actually create the ability with the following code:

with EyePopSdk.dataEndpoint(api_key=EYEPOP_API_KEY, account_id=EYEPOP_ACCOUNT_ID) as endpoint:
   for ability_prototype in ability_prototypes:
       ability_group = endpoint.create_vlm_ability_group(VlmAbilityGroupCreate(
           name=ability_prototype.name,
           description=ability_prototype.description,
           default_alias_name=ability_prototype.name,
       ))
       ability = endpoint.create_vlm_ability(
           create=ability_prototype,
           vlm_ability_group_uuid=ability_group.uuid,
       )
       ability = endpoint.publish_vlm_ability(
           vlm_ability_uuid=ability.uuid,
           alias_name=ability_prototype.name,
       )
       ability = endpoint.add_vlm_ability_alias(
           vlm_ability_uuid=ability.uuid,
           alias_name=ability_prototype.name,
           tag_name="latest"
       )
       print(f"created ability {ability.uuid} with alias entries {ability.alias_entries}")

‍

That’s it! To run the prompt against an image here is some sample evaluation code:

from pathlib import Path


pop = Pop(components=[
   InferenceComponent(
       ability=f"{NAMESPACE_PREFIX}.structured-OCR.read-license-plate:latest"
   )
])


with EyePopSdk.workerEndpoint(api_key=EYEPOP_API_KEY) as endpoint:
   endpoint.set_pop(pop)
   sample_img_path = Path("/content/sample_img.png")
   job = endpoint.upload(sample_img_path)
   while result := job.predict():
      print(json.dumps(result, indent=2))


print("Done")

‍
After running the evaluation you can see what the model described and compare it to your source of truth. With this, you can improve your prompts and thus improve your accuracy.