You've just built and launched an anonymous photo sharing app (WiSaw) which has tons of cool pictures.
Now you want to start promoting it to masses to benefit from economy of scale principles.
But, most of your photos do not have text associated with it -- you've got a problem. The search crawlers dwell on text. Text is what they eat. Text is what they want. No text -- no ranking.
One thing you should do is to make it easy for people to comment on photos. This approach always works. The only problem is -- it takes time to build volume. People have to first find your cool app, and then they can start posting photos and commenting on them. In order for users to find your cool service -- photos must have meaningful text already available for crawlers. This is a classical catch 22 dilemma.
So, what is the easy way out of it? How do you bootstrap your service so that it starts appearing in searches?
Wouldn't it be nice, if the computers "understood" what's on your photo and turned it into text? Well, this is exactly what Amazon Rekognition is all about.
It was super easy to add image recognition to the product, so that every new photo would get "rekognized".
As the result "Google Search Console" reports jump in number of indexed pages increased by factor of 20! It went from 90 indexed pages up to 1,779 in few short weeks (yes, it takes time for crawlers to crawl).
Amazon Rekognize service offers recognition in few categories:
We only used the "Object and scene detection", "Text in image", and "Image moderation". What is the likelihood that someone will snap a selfie with a celebrity? Also, since we are building anonymous photo sharing, we felt it's not the best idea to start detecting faces. And after all, you have to pay for every recognition type you invoke separately. Not a whole lot, but it may add up. To process almost 5k images in 3 mentioned categories, we had to pay around $10.
So, how well does "image rekognition" works at this stage?
Let's look at some examples:
Text detection works quite reliable. It will recognize most of the non hand written text. Obviously, adding artistic styling to the text will decrease the accuracy. Also, this service is not to be confused with OCR type of recognition. If you want to convert photos of pages of text, it will not do what you may expect. It will only recognize up to 50 words, which should be enough for what we need -- providing some textual context for web crawlers.
Objects and scenes detection works well for simple objects. Once you start combining different things, it begins to struggle. For instance, look at the photo below:
It properly recognized that it's some kind of a food. But, it really thinks that it's something sweet.
Note, the "rekognize service" returns a certainty for every guess it makes. We use that to show more certain guesses in larger font. As you can see, the certainty decreases top to bottom. Hopefully, this will also trick the crawlers into ranking pages with "richer" content higher.
Content moderation is a separate category. When we started to build our app (WiSaw), one of the biggest challenges we had to overcome was to get our app approved on the app store. The iTunes would simply refuse to publish the app, because they require content to be moderated. To overcome this, we leaned on community moderation, rather than on company sifting through all the content. We added a "Delete" button to every content you see, even if it was not content you provided. Anyone can post anything, and anyone can delete anything as well. This little trick was sufficient to get passed the stringent rules of Apple. But even then, we want to believe that people are always driven by good intentions, and. if someone starts posting something nasty, someone else will delete it in a snap. But what if we are wrong? Having ability to auto filter content based on ML and AI is a super cool feature we'd like to use some day. Unfortunately, at this stage, it's not very reliable. It may help you to search for content that may potentially need to be moderated. But relying on this service to delete user's photos, we may end up with missing images which would make our users unhappy. Here are few examples of false positives:
Fruit pie is "rekognized" as "Explicit Nudity"? Really?
Here is one more:
Adult toys?
Nudity when it's just a cup holder?
What would you think about two ladies peacefully bird watching through their binoculars in Arboretum park? AI thinks this post contains "Violence" and "Weapon Violence":
So, it's getting there, some day. For now -- it's only good for providing some text over images, just enough to get crawlers excited about your anonymous photo sharing app.