Apresentação do PowerPoint - NMGIC
22
SHADOWS REMOVAL IN HIGH RESOLUTION REMOTE SENSING IMAGES USING LOCAL INPAINTING STRATEGY Samara C. de Azevedo¹ Guilherme P. Cardim¹ Wallace Casaca¹ Erivaldo Silva¹ Ramesh P. Singh² Supported by: 1 2
-
Upload
khangminh22 -
Category
Documents
-
view
1 -
download
0
Transcript of Apresentação do PowerPoint - NMGIC
SHADOWS REMOVAL IN HIGH RESOLUTION REMOTE SENSING IMAGES USING LOCAL INPAINTING STRATEGY
Samara C. de Azevedo¹ Guilherme P. Cardim¹ Wallace Casaca¹
Erivaldo Silva¹ Ramesh P. Singh²
Supported by:1 2
Remotely sensed data
Remote sensing, as a part of general sensing, computing science andinformation technologies, is greatly affected by universal development trends.
Multiple Platform
Multisensory Systems
-Imaging performance-Sensor agility-Resolutions
Improvements:
Pecora 20 - Observing a Changing Earth; Science for Decisions— Monitoring, Assessment, and Projection, November 13-16, 2017 Sioux Falls, SD
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
New sensors developmentm
30
10
4
0.5
1
80
0.3 http://www.satimagingcorp.com
1970 1980 1990 2000 2010 20??2017
Urban monitoring and planning;
Agricultural management;
Land cover mapping;
Environmental studies;
Applications:
The combination of finer resolution, low sun elevation and tall buildingsmay lead to many undesirable artifacts, e.g., shadows.
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
Shadows
Reduction of information Corruption of biophysical parametersNDVI
Drawbacks:
Hampers the accuracy of cartographic feature extraction
Disturbs orignated in Computer Vision applications
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
Liu e Yamasaki (2009) Zanin (2012)
Sanin et al. (2012)
Shadow Effects Removal
Proposal: a novel approach that addresses the problem of shadow removal in VHRimages from urban areas.
Goal: brightness restoration by using information from the surrounding pixels
Multitemporal replacement Exemplar-based inpainting methods
Highlights:
• Automatic detection of shadows based on contextual and spectral image features;
• Inpainting strategy from a shadow guidance mask and multispectral bands as a concise andimproved filling order mechanism to recovering large cast shadows regions;
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
Shen et al. (2015)
Pipeline overview
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
Input DatasetPan + Multi
PREPROCESSING
Radiometric correction Pansharpening
SHADOW DETECTION
Shadow Candidates Generation Shadow Detection Refinement Shadow Mask
SHADOW REMOVAL
Cartoon Image Computation Local Inpainting Shadow completation
Preprocessing
Radiometric correction
Pansharpening
methodCC RMSE ERGAS UIQI
Gram-Schmidt 0.8643 0.0025 0.0901 0.8642
-Insertion of spatial details
-Visual quality improved
-TOA reflectance
-Haze separation
Quality Metrics
Pansharpening
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
Obtaining the Shadow Mask
CONTEXTUAL
Area-closing
( )iB
if
the output is the sum of all filtered input threshold images with dark structures w.r.t. parameter λ
SATURATION-VALUE DIFFERENCE INDEX (NSDVI) MASK
Area parameter
SPECTRAL
( ) ( )BTH f f f
Extract dark patterns by taking the residuals of theclosing, and the original image, by a morphologicaltransformation called top-hat
L=40
L=10
L=20
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
Obtaining the Shadow Mask
Definition of area parameter
NSDVI LABELED
Area estimation
CONNECTED OBJECTS RGB-> HSV
SATURATION
VALUE
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
Shadow Mask Refinement
TOP-HAT
Otsu binarization NDVI
Multilevel analyses Adaptive filtering -> non requirement
for SE with prefixed sizes and shapes Automatic selection scheme Low false detections
Advantages
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
Shadow removal
Key concept: “Combining anisotropic diffusion + transport equation + texturesynthesis as a effective approach for recovering shadow pixels”
Inpainting Strategy
(t) (t)(t) (t)
(t)(1 )( ),
f fg f div g f f
t f
Getting a reference image: an anisotropic diffusion filter is applied to better capture the structures of interest from the target image f
g is an edge detection function (as in [2]); f is the input image; f(t) is the scaled version of f.
Equation (1) is numerically solved by using FinitieDifference schemes
(1)
Output: component u is a smoothed image that holds the geometric structures andhomogeneous parts of f.
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
Shadow removal
Filling order process
A Inner product-derived metric which is based on the transport equation (the BSCBmethod (Bertalmio et al., 2000)), is applied to u so that the filling order of theshadow pixels is carefully chosen in terms of their neighborhood non-shadow pixels
( ), i i i(p ) = R(p ) C p p
MATHEMATICALLY
( ) ( ) , ppp p
p
du
R p u du
R: accounts for the direction of the image structures. C: keeps the coherence during the completion process.
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
Shadow removal
Mask completion
Shadow Region
Most similar patch of pixels from the dynamic region ΛΩp (orange square) are allocated in the shadow covered area Ω
2 2( , )
U U
Up q
d p qp q
Cartoon-based metric:
Multispectral bands are embedded into the distance calculation
2:U
Tp p Up
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
Inpainting strategyExample:
(a) (b) (c)
(d) (e) (f)
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
Inpainting strategy
Example:Holds the patch with more unfilled pixels
low C
high C
Moderate C
(g) (h)
Dataset
10th World’s largest urban area
WorldView-2 (WV2) VHR
imagery (July 06, 2012 )
Pan (50 cm)
Multi (2 m)
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
Shadow detection
Producer’s accuracies
(%)
User’s accuracies
(%)
Overall accuracy
(%)
93.51 81.72 90.27
Performance evaluation
Input WV2 image Shadow mask result by our approach
Result overlaid with original image
Subset 1
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
Shadow detection
Producer’s accuracies
(%)
User’s accuracies
(%)
Overall accuracy
(%)
94.29 79.81 86.83
Input WV2 image Result overlaid with original image
Performance evaluation
Subset 2
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
Shadow mask result by our approach
Inpainting result
Input subset 1 our strategy
Input subset 2 our strategy
Improved useful areas
Good visual coherence
Building rooftops, Highway and
cars were properly recovered
Mislead information
Does not maintain the
geometric configuration
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
Shadow removal
Inpainting Quality: proposed methodology x a state-of-the-art approach
Result by our approach Histogram Matching (HM) HM reprocessed
White edgesPenumbra / degenerative effectsPoor visual quality
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
Future Works
Main Remarks
The proposed framework accomplishes the task of shadow removal while stillpromoting a good visual distinction of the image elements.
anisotropic diffusion + a new filling order mechanism + exemplar-based completion
Robust apparatus to properly repair large shadows areas combining:
Good trade-off between visual quality and low computational cost in a complex urban environment by only taking as input a single scene.
Mitigate discontinuous patches Apply the methodology to other existing sensors to better understand the tuning effects
of several parameters under urban environments. Assessing the repair results via numerical analysis and unsupervised classifications.
INTRODUCTION METHODOLOGY EXPERIMENTAL RESULTS CONCLUSION
High coherence and penumbra-effects mitigation.
