The most important scientific articles are listed here. It's a hand-curated collection of the most recent AI and data science developments, organized chronologically with a link to a more in-depth article.

AI-powered neuroprosthetic hand

Using this AI-powered nerve interface, the amputee can control a neuroprosthetic hand with real skill and intuitiveness.

Deep learning-based neural decoders have become the most popular way to make neuroprosthetic hands easy to use and flexible. Still, deep learning hasn't been used much in clinical settings because it takes a lot of computing power. However, new developments in edge computing devices could help solve this problem. The researchers show how they made a neuroprosthetic hand controlled by deep learning. The neural decoder is of a recurrent neural network (RNN). It runs on the NVIDIA Jetson Nano, a small but powerful platform for deep learning inference at the network's edge. This research makes it possible to use the neuroprosthetic hand as a portable, self-contained unit that can control the movements of each finger in real-time.

A transradial amputee with implanted intrafascicular microelectrodes tests the peripheral nerve signals (ENG). The experiment results show that the system can provide reliable, high-accuracy (95–99%), and low-latency (50–120 msec) control of each finger's movement in the lab and real-world settings.

Modern edge computing platforms allow deep learning-based neural decoders to control neuroprostheses as an independent system. Furthermore, this work helps pave the way for deep neural networks in clinical applications. Finally, this research is the basis for a new class of wearable biomedical devices with AI built-in.

Paper: Portable, self-contained neuroprosthetic hand with deep learning-based finger control

Total relighting

The researchers have developed a new system for relighting portraits and replacing backgrounds. This system keeps high-frequency boundary details and accurately recreates how the subject looks when lit by a new light source. This research makes it possible to create realistic composite images for any scene. Their method includes alpha matting, relighting, and compositing to estimate the foreground.

The researchers show that we can handle each stage in a sequential pipeline without priors (like a known background or lighting) and any special acquisition techniques, using only a single RGB portrait image and a new, target HDR lighting environment as inputs. Researchers train their model with relit portraits of people taken in a light stage computational illumination system, which records multiple lighting conditions, high-quality geometry, and accurate alpha mattes.

The researchers introduce a new per-pixel lighting representation in a deep learning framework to do realistic relighting for compositing. This research explicitly models the diffuse and specular parts of appearance, making it possible to relight portraits with convincingly rendered non-Lambertian effects like specular highlights. Multiple tests and comparisons show that the proposed method works well when applied to images taken in the real world.

Paper: Total relighting: learning to relight portraits for background replacement

LASR

Reconstructing rigid structures in 3D from a video or a group of still images has come a long way. But it's still hard to rebuild nonrigid structures from RGB inputs because they aren't well defined. Moreover, while template-based approaches, like parametric shape models, have done a great job of modelling the "closed world" of known object categories, they don't do as well in the "open world" of new object categories or outlier shapes.

In this paper, the researchers describe how to learn 3D shapes from a single video that doesn't use a template. Instead, it uses an analysis-by-synthesis method that forward-renders an object's silhouette, optical flow, and pixel values to compare with video observations. This research makes gradients that can change the camera, shape, and motion parameters. Our method reconstructs nonrigid 3D structures from videos of people, animals, and objects of unknown classes without using a shape template for each category.

The researchers talk about LASR, a way to reconstruct articulated shapes from a single-view video without using a template. LASR accurately reconstructs individual objects from different categories (like a person, a camel, a dog, a bear, etc.) without using shape templates for each type. This research makes it useful in a wide range of situations. Furthermore, the researchers hope that LASR will make it possible to make more progress in reconstructing shapes with joints.

Paper: LASR: Learning articulated shape reconstruction from a monocular video

Click here for the code.

Image source: Unsplash