Half-baked ideas: OCR VM console to diagnose state and errors

For more half-baked ideas, see the ideas tag.

These needs a better name, but the idea is simple. Take a screenshot of your guest’s graphical console, and OCR it in order to diagnose the VM’s state, any errors, etc.

Case 1: VM error on boot
errors

Case 2: VM has reached graphical login screen
gnome3

OCR would analyze the screenshot, find everything that looks like text and convert it to text, allowing for simple regular expressions to be used to identify VM state.

Notes:

  1. Strictly speaking in the first case you could read the screen text directly out of the text framebuffer, but OCRing is more general and handles the second case.
  2. virt-dmesg could also solve the first case, assuming virt-dmesg wasn’t broken.

7 Comments

Filed under Uncategorized

7 responses to “Half-baked ideas: OCR VM console to diagnose state and errors

  1. I’ve also thought about this recently. Initial testing using gocr and tesseract didn’t lead to any promising results.
    But definetly using OCR might be a better alternative (because it’s text and regex based) then using computer vision where you need to maintain templates to match anything specififc on the screen.

    We (oVirt Node) are using the first approach for functional test of our TUI.

  2. Chris

    Yeah, this sort of thing would rock for Oz. If there is a guest VM failure, Oz already takes a screenshot[1] and puts it on disk for the user, but doing an OCR on the resulting image would make the error messages much better.

    [1] Using the libvirt screenshot mechanism.

  3. Matthias Clasen

    We are doing this sort of automated ‘did it boot ?’, ‘did login work ?’ testing in ostree (http://build.gnome.org/#gnome-ostree) by looking for expected messages in the journal. Using message ids lets us do this pretty reliably.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s