# SCH: Scene-Aware AR Systems for Low Vision People in Activities of Daily Living

> **NIH NIH R01** · UNIVERSITY OF WISCONSIN-MADISON · 2024 · $352,291

## Abstract

Uncorrectable vision loss, a.k.a. low vision, can significantly impact activities of daily living (ADLs),
affecting indepe ndence, health, and social well-being . Magnifiers are the most commonly used low vision
aids, however, they largely reduce visual field. Augmented reality (AR) glasses present a unique
opportunity f or low vision technology by intelligently recognizing the environment and enhancing users'
vision based on their contexts. However, existing AR solut ions mainly rely on image processing
techniques, generating pure pixel-level augmentations to the whole scene, such as pixel remapping for
visual field loss, and recoloring based on distance. These solutions arbitrarily alter the user's full visual
field with no semantic understanding of the scene, distorting users' natural vision and diminishing
important visual details. As a result, they ca nnot effectively support complex ADLs that involve constant
motion and object interactions, such as cooking in the kitchen, or safely navigating a crowded street. The
overall objective of this proposal is to fundamentally advance low vision technology by leveraging
state-of-the-art Al techniques and presenting unobtrusive scene-aware AR augmentations that are
tailored to users' environments and visual abilities. Contextualized in two key ADLs (meal preparation,
outdoor mobility), the research will investigate how to recognize and augment visual elements in AR from
three fundamental dimensions via three specific aims: (A1)
E
nhancing perception of visual salience, (A2)
E
nhancing object affordance for safe and efficient interactions, (A3)
E
nhancing awareness of
environmental dynamics. In each aim, the research will contribute to both HCI and Al by designing
scene-aware AR augmentations with low vision users and rehabilitation professionals, and developing
egocentric video datasets and Al models to enable fast and robust scene recognition in AR. In the final
year, a holistic AR system will be developed and evaluated for real world impact and limitations via field
studies (A4). With the Al-powered AR systems f or low vision, the project will contribute to the mission of
N
EI
by fundamentally transforming conventional low vision aids and improving independence and quality
of life for low vision people in various challenging ADLs. The involvement of low vision participants and
rehabilitation professionals throughout the research will also increase their exposure to state-of-the-art
technolog y, potentially increasing the technology acceptance and adoption in low vision rehabilitation.
RELEVANCE (See instructions):
The proposed research is relevant to public health because it investigates how to design and develop
Al-powered AR systems to enhance low vision users' visual perceptions, empowering them to navigate
activities of daily living with greater safety, efficiency, and independence. The project's contribution to low
vision technology directly aligns with the mission of N
EI
, which emphasizes t...

## Key facts

- **NIH application ID:** 11063313
- **Project number:** 1R01EY037100-01
- **Recipient organization:** UNIVERSITY OF WISCONSIN-MADISON
- **Principal Investigator:** Yapeng Tian
- **Activity code:** R01 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2024
- **Award amount:** $352,291
- **Award type:** 1
- **Project period:** 2024-09-30 → 2028-08-31

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/11063313

## Citation

> US National Institutes of Health, RePORTER application 11063313, SCH: Scene-Aware AR Systems for Low Vision People in Activities of Daily Living (1R01EY037100-01). Retrieved via MORVS 2026-10-25 from https://api.morvs.ai/grant/nih/11063313. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*