Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2010 Sep 16:9:52.
doi: 10.1186/1475-925X-9-52.

Improved content aware scene retargeting for retinitis pigmentosa patients

Walid I Al-Atabany  1 , Tzyy Tong, Patrick A Degenaar
Affiliations

Improved content aware scene retargeting for retinitis pigmentosa patients

Walid I Al-Atabany et al. Biomed Eng Online. .

Abstract

Background: In this paper we present a novel scene retargeting technique to reduce the visual scene while maintaining the size of the key features. The algorithm is scalable to implementation onto portable devices, and thus, has potential for augmented reality systems to provide visual support for those with tunnel vision. We therefore test the efficacy of our algorithm on shrinking the visual scene into the remaining field of view for those patients.

Methods: Simple spatial compression of visual scenes makes objects appear further away. We have therefore developed an algorithm which removes low importance information, maintaining the size of the significant features. Previous approaches in this field have included seam carving, which removes low importance seams from the scene, and shrinkability which dynamically shrinks the scene according to a generated importance map. The former method causes significant artifacts and the latter is inefficient. In this work we have developed a new algorithm, combining the best aspects of both these two previous methods. In particular, our approach is to generate a shrinkability importance map using as seam based approach. We then use it to dynamically shrink the scene in similar fashion to the shrinkability method. Importantly, we have implemented it so that it can be used in real time without prior knowledge of future frames.

Results: We have evaluated and compared our algorithm to the seam carving and image shrinkability approaches from a content preservation perspective and a compression quality perspective. Also our technique has been evaluated and tested on a trial included 20 participants with simulated tunnel vision. Results show the robustness of our method at reducing scenes up to 50% with minimal distortion. We also demonstrate efficacy in its use for those with simulated tunnel vision of 22 degrees of field of view or less.

Conclusions: Our approach allows us to perform content aware video resizing in real time using only information from previous frames to avoid jitter. Also our method has a great benefit over the ordinary resizing method and even over other image retargeting methods. We show that the benefit derived from this algorithm is significant to patients with fields of view 20° or less.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Algorithm flow. This diagram illustrates the major steps in our algorithm, the top raw shows the processing in the horizontal direction and the bottom raw shows the vertical one.
Figure 2
Figure 2
Basic algorithm outcomes. The output from each stage in our Seam Shrinking algorithm, when applied to two frames of video file. (a) is the two input images, (b), (c), (d) the spatial, spatio-temporal and modified importance matrix, respectively, (e), (f) the shrinkage map when using the unmodified and modified importance matrix, respectively, (g), (h), and (i) the retargeted image when using just interpolation, unmodified and modified shrinkage map, respectively.
Figure 3
Figure 3
The effect of discontinuity of rows. (a) A 400 × 650 image. The image is retargeted to 400 × 400 by shrinking each row independently (b), causing a Zig-Zag, and when preserving row continuity (c).
Figure 4
Figure 4
Illustration of the edge flipping problem. When the shrinkability value exceeds 1, the edge of the shrinkage pixel will move leftward past its left-hand edge. The number above each arrow indicates the value by which the pixel should be compressed.
Figure 5
Figure 5
Snapshot of our synthetic test video. Showing three text boxes and blue alignment marker in the four directions; top, down, right and left.
Figure 6
Figure 6
A comparison between the performance of seam carving, shrinkability, and our method by measuring the three parameters: The OCR (top row), the compressibility (middle row) relative to the ordinary interpolation, and the Zig-Zag effect. The left column shows the performance when σ of the added Gaussian noise (simulation to the contrast between the foreground and background) is 0.025, and the right column when σ is 0.45.
Figure 7
Figure 7
Comparison between the retargeting of the frame by 50% of its original size when using. The shrinkability algorithm (a), the seam carving algorithm (b), and our seam shrinking algorithm (c).
Figure 8
Figure 8
The effect of increasing σ of the added Gaussian noise (to simulate the variation on contrast between the foreground and background). When 25% compression rate from the original size is required (left column) and 62% compression rate (right). The first row is for the OCR measurement, the second row is for the compressibility measurement, and the last row is for the Zig-Zag or unalignment measurement.
Figure 9
Figure 9
A comparison between different image retargeting techniques and our method. From left to right: original, interpolation, shrinkability, seam carving, and our seam assisted shrinkability method. All the images are retargeted to 62% of the original size.
Figure 10
Figure 10
The efficiency of the seam assisted shrinkability method on search time. We simulated the effect of tunnel vision by asking users to wear blacked out goggles with apertures corresponding to 11° and 22° FOV. Greater improvement on search time improves as the tunnel vision worsens. Mid-Late stage RP typically expresses tunnel vision in the 5° to 20° range. The error bars represent the standard error of the data.
Figure 11
Figure 11
Recognition variation with spatial compression. The efficiency of retargeting the images using our seam assisted shrinkability method compared to the ordinary resizing method in recognizing objects, when scenes were compressed into 10%, 20%, 40% and 50%, respectively, of their original sizes. The error bars represent the standard error of the data.
Figure 12
Figure 12
The efficiency of detecting objects. The efficiency of detecting objects when retargeting dynamic scene into 40% of the original size using our seam assisted shrinkability method compared to the original size. The error bars represent the standard error of the data.
Figure 13
Figure 13
The efficiency of recognizing actions. The efficiency of recognizing actions when compressing the dynamic scene into 25%, 35, and 45% of the original size, using the ordinary resizing method and our method. The error bars represent the standard error of the data.
Figure 14
Figure 14
Subjective preference test. The preference test is between our method and the seam and shrikability methods. The test was done on 14 subjects showed to them 6 retargeted videos using the three methos.
Figure 15
Figure 15
Subjective preference test. The preference test is between our method and the ordinary resized method.
Figure 16
Figure 16
Limitation of the algorithm for busy environment. (a) Snapshot of a busy video for guys playing with basketball in a very small area. (b) Shows a 25% compression on it, when compressed linearly (right) and with our retargeting algorithm (left).
Figure 17
Figure 17
Samples from the trial. Samples of the images used in the first section of the test (a). And in the second section (b), where the image is compressed to 10% (right) and 20% (left) of the original size, respectively. The left image in each pair of images in (b) is the one compressed linearly and the right one is the retargeted using our method.
Figure 18
Figure 18
Snapshots from the dynamic scenes. Snapshots from the second video file in the third section of the test, before (a) and after (b) retargeting, respectively. Snapshots from the video used in the forth section of the test, where the video is compressed into 25% of its original size using linear resizing (c) and our retargeting method (d).
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Foster A, Resnikoff S. The impact of Vision 2020 on global blindness. Eye. 2005;19:1133–1135. doi: 10.1038/sj.eye.6701973. - DOI - PubMed
    1. Hamel C. Retinitis pigmentosa. Orphanet Journal of Rare Diseases. 2006;1:40. doi: 10.1186/1750-1172-1-40. - DOI - PMC - PubMed
    1. Vargas-Martín F, Peláez-Coca MD, Ros E, Diaz J, Mota S. A generic real-time video processing unit for low vision. International Congress Series. 2005;1282:1075–1079. doi: 10.1016/j.ics.2005.05.107. - DOI
    1. Fullerton M, Peli E. Post Transmission Digital Video Enhancement for People with Visual Impairments. J Soc Inf Disp. 2006;14:15–24. doi: 10.1889/1.2166829. - DOI - PMC - PubMed
    1. Wolffsohn JS, Mukhopadhyay D, Rubinstein M. Image enhancement of real-time television to benefit the visually impaired. Am J Ophthalmol. 2007;144:436–440. doi: 10.1016/j.ajo.2007.05.031. - DOI - PubMed

Publication types

-