Posts Tagged ‘uima’

Text analytics in BlueMix using UIMA

Sunday, April 13th, 2014

In this post, I want to explain how to create a text analytics application in BlueMix using UIMA, and share sample code to show how to get started.

First, some background if you’re unfamiliar with the jargon.

What is UIMA?

UIMA (Unstructured Information Management Architecture) is an Apache framework for building analytics applications for unstructured information and the OASIS standard for content analytics.

I’ve written about it before, having used it on a few projects when I was in ETS, and on other side projects since such as building a conversational interface to web pages.

It’s perhaps better known for providing the architecture for the question answering system IBM Watson.

What is BlueMix?

BlueMix is IBM’s new Platform-as-a-Service (PaaS) offering, built on top of Cloud Foundry to provide a cloud development platform.

It’s in open beta at the moment, so you can sign up and have a play.

I’ve never used BlueMix before, or Cloud Foundry for that matter, so this was a chance for me to write my first app for it.

A UIMA “Hello World” for BlueMix

I’ve written a small sample to show how UIMA and BlueMix can work together. It provides a REST API that you can submit text to, and get back a JSON response with some attributes found in the text (long words, capitalised words, and strings that look like email addresses).

The “analytics” that the app is doing is trivial at best, but this is just a Hello World. For now my aim isn’t to produce a useful analytics solution, but to walk through the configuration needed to define a UIMA analytics pipeline, wrap it in a REST API using Wink, and deploy it as a BlueMix application.

When I get a chance, I’ll write a follow-up post on making something more useful.

You can try out the sample on BlueMix as it’s deployed to bluemix.net

The source is on GitHub at github.com/dalelane/bluemixuima.

In the rest of this post, I’ll walk through some of the implementation details.

(more…)

Tags:, , ,
Posted in code | 2 Comments »

Avoiding monitor contention in Java’s Double parseDouble

Saturday, November 30th, 2013

Overview

You can call Double.parseDouble in Java to convert String representations of numbers like “1.234567” into a number representation.

I needed to do this, a lot of times, from a lot of threads. And it was horrendously slow.

In this post, I’ll explain why and what I did about it.

Background (skip this if you don’t care why I had this problem!)

I’ve mentioned UIMA before : an Apache framework for doing text analytics, that I use at work. One of the ways that it stores and moves around units of work is in XML files (called CAS files).

For a particular task at work, I will have a lot of these. Thousands of them. I need to deserialize these, and parse and process the contents. The contents includes scores from the various analytics operations that are done on the contents of the CAS:

<myElement
    myRawScore="1.2345678"
    myThisScore="2.46801357"
    myThatScore="1.35792468"
    ...

Thousands of XML files, each containing several thousand numbers in String form.

As part of the deserializing the CAS files, the UIMA library (specifically org.apache.uima.cas.impl.CASImpl) was calling Double.parseDouble 500,000,000 times or more.

I’ve got 64 processor cores and lots of memory, so I kicked off 64 threads – each one processing an XML CAS file at a time.

This took *ages*.

(more…)

Tags:,
Posted in code | Comments Closed

Conversational Internet : A prototype

Wednesday, September 12th, 2012

tl;dr

We’ve built a prototype to show how we could interact with the Internet using a command-driven approach.

  • A screen reader, but one that uses machine learning and natural language processing, in order to better understand both what the user wants to do, and what the web page says.
  • One that can offer a conversational interface instead of just reading out everything on the page.

It’s a proof-of-concept, but it’s an exciting idea with a lot of potential and we’ve got a demo that shows it in action.

I wrote yesterday about what it was like going to the BBC to talk about a project I’ve been working on this summer. I didn’t talk about the project itself. Here’s an overview.

(more…)

Tags:, ,
Posted in ibm, tech | 4 Comments »

Using UIMA-AS to run UIMA annotators in parallel

Friday, August 24th, 2012

Overview

UIMA stands for Unstructured Information Management Architecture. It’s an Apache technology that provides a framework and standard for building text analytics applications. I’ve mentioned it before.

In this post, I want to talk about an area of UIMA which isn’t covered well in the documentation.

I couldn’t find practical getting-started instructions for running UIMA-AS annotators in parallel. In this post I want to discuss why you might want to do it, and share some simple sample code to show how.

Background – the UIMA pipeline

UIMA provides a framework for managing a text analytics application. You break up the analytics functionality into discrete pieces called annotators. UIMA takes care of moving a text document through an analytics engine: a pipeline containing a series of annotators.

A document goes in one end of the pipeline, passes through a number of annotators, each of which adds some metadata to the document. What comes out the other side of the pipeline is an annotated copy of the document.

By default, you get UIMA to run these annotators one at a time – one after another.

Background – annotators in parallel

What if your annotators are quite slow – perhaps they take several seconds to run?

If there is no dependency between any or all of your annotators, then maybe running them one at a time isn’t the most efficient approach.

You can run all of them at the same time, in parallel. UIMA will merge the output from all of the annotators into a single annotated document.

My sample code

I’ve written two sample UIMA apps. Each demonstrates one of these approaches, to compare and contrast.

They are divided into three eclipse projects. You can import them into an eclipse IDE.

The UIMA eclipse plugins are very helpful if you want to make changes to the XML configuration files, but they’re not essential. If you want them, there are instructions on how to install them at uima.apache.org.

I’ve added comments to the sample code to explain how the apps work, but I’ll give an overview here.

(more…)

Tags:, , ,
Posted in code | 1 Comment »

Using JMX to monitor UIMA running in a servlet

Wednesday, August 1st, 2012

Overview

A quick howto for if you’re running UIMA in a servlet, and want to be able to monitor your AE performance using JMX

Background

I’ve mentioned JMX before. Basically, a Java app can expose information and methods through a standard interface. Tools like jconsole, which come with Java, can then be used to monitor and administer the Java app.

UIMA (Unstructured Information Management Architecture) is an Apache project, providing a standards-based way to perform analytics on unstructured text. It hosts a pipeline of annotators: individual components each performing a specific text analytics task. As a document moves down the pipeline UIMA runs each of the annotators on the document. Each annotator adds it’s own annotations for the things it looks for in the text.

UIMA and JMX

UIMA supports JMX. UIMA registers an MBean for each annotator, letting you see the performance info for each annotator. In a pipeline of several annotators, it lets you see (amongst other things) how much time your document is spending in each annotator.

jconsole

In a stand-alone UIMA application, you basically get this for free. Start the application with the standard Java -D property for enabling JMX:

-Dcom.sun.management.jmxremote

It is ready to let jconsole connect to it.

(more…)

Tags:, , , ,
Posted in code | Comments Closed