The first image in your mind in response to the word 'classroom' will, in most likelihood, be a blackboard. That's because it is perhaps the most powerful tool in the hands of a teacher, which makes an indelible mark in the minds of all students. The blackboard is where a teacher draws, writes and annotates while explaining complex concepts. Given that the human mind remembers pictures better than, this technique is invaluable to every student's learning process.

In today's digitised world, classrooms are going digital, which requires STEM content to be reimagined for desktop or handheld screens. The challenge for every STEM content developer is to bring the same amount of clarity as provided by a teacher with a blackboard by supporting diagrams and equations with lucid explanations on-screen. 

The problem becomes exponentially severe when developing content for students with accessibility needs - primarily visual and cognitive. For many STEM topics and subjects like accounting, diagrams, graphs, and tables are essential to understanding concepts. In chemistry, for example, a bulk of the subject matter is represented through equations and line diagrams. 

According to Continual Engine founder Mousumi Kapoor “Over 700 million students have some sort of visual disability. Studies have shown that over 80% of students can’t pursue STEM subjects because the course content is not accessible

Publishers and teachers have been trying to overcome this hurdle by including an 'alt-text or alternative text. Alternative text is a comprehensive textual description of an image, diagram, chart, table, or graph. In the case of a graph the alt-text would describe the axes, the interval between points in each axis, the coordinates of essential points, shape and trend of the graph. 

The only issue is, creating alternative text is an expensive and time-consuming affair. That is because until now, the only reliable way of doing it was through the manual intervention of a subject matter expert (SME). Even screen readers cannot parse images effectively. 

Employing SMEs to describe every image in a textbook or course is a resource-intensive task. Typically, it takes a cycle time of 2 to 3 months for a single book worth of images to be manually authored to alt-text and may cost anywhere between $10,000 and $100,000.

Invicta, an AI alt-text authoring platform from Continual Engine, might be the solution to this problem.

According to Mousumi Kapoor "Using AI to automate the process of describing images, equations, graphs, tables etc. into high-quality, accurate descriptions, will reduce cycle time and cost associated with alt text."

Using an AI-powered system to automate alt-text authoring can lead to a 50% reduction in cycle time and a 60% reduction in costs of creating alt-text. 

Invicta is a very sophisticated solution since each subject requires very different parsing techniques. For example, equations and graphs in Mathematics require very different approaches when compared to parsing line equations and aromatics and line diagrams in Chemistry.

To translate technical images like equations and graphs from various STEM subjects into highly detailed, standardised descriptions, Invicta needs 6 Steps.

  • Image Capture: Invicta accepts images across multiple file-formats, including jpeg, png, and eps.
  • Pre-Processing: Images are then optimised and denoised to be analysed by the AI. This optimisation and correction process includes binarization, skewing, segmentation and removing outliers.
  • Image Processing: Invicta uses deep neural network-based image processing architecture to extract essential features from the images. This is done using various techniques, including computer vision, object detection, CNN, RNN and seq 2 seq modelling, to name a few. 
  • Feature Transformation: The features extracted are then transformed into standardised algorithm readable structures such as JSON, SMILE and LaTeX.
  • Parsing: An algorithm based parser will then transform the features described into text.
  • Human in the Loop: Finally, an SME will check the machine's output and validate the result to maintain accuracy.

The ability to fully transpose images into text represents a paradigm shift in educational content publishing. 

Firstly, it makes STEM subjects accessible for students with cognitive and visual accessibility needs. A screen reader can easily read the alt-text generated by Invicta. Since the solution is automated, publishers can pursue alt-text inclusion at scale, making it feasible for many students to pursue their STEM goals.

Secondly, Invicta's solutions benefit teachers, especially in the digital new-normal world of post-2020. The alt-text descriptions that Invictus generates are both highly detailed and highly standardised. This means that teachers can use these texts as ready-reference with which to teach students. Also, since the text is highly standardised, descriptions of subject-critical diagrams will not be wary of a teacher to teacher. 

Finally, Invicta becomes an essential keystone in the push towards modular digital content publishing. Publishers are starting to move away from the basic digital transposition of educational content like PDF textbooks. Today, education publishers are looking to publish content in sophisticated modular content-aware packages that can be staked into bespoke bundles. The hurdle so far has been technical images and diagrams. Invicta adds that this capability can become a crucial step in the digital content publishing pipeline. 

There is no doubt that an AI tool like Invicta can change the landscape of STEM learning. The technology makes STEM subjects and their complex concepts more accessible to a much larger group of students, making the stream more inclusive than it's ever been.

Moreover, the capabilities it adds to publishers who use it cannot be understated. It is quite possible that shortly, all publishers will be using the tool, which in addition to improving inclusivity also will become a mainstay in their digital content push.

Want to publish your content?

Publish an article and share your insights to the world.

ALSO EXPLORE

DISCLAIMER

The information provided on this page has been procured through secondary sources. In case you would like to suggest any update, please write to us at support.ai@mail.nasscom.in