Deep learning model to detect and segment trees in high-resolution imagery. A brief summary of the item is not available. Add a brief summary about the item.
Deep learning package
by
Item created: May 18, 2023 Item updated: Jan 1, 2025 Number of downloads: 44,461
Description
This deep learning model is based on DeepForest and has been trained on data from the National Ecological Observatory Network (NEON). The model also uses Segment Anything Model (SAM) by Meta.
Using the model
Follow the guide to
use the model. Before using this model, ensure that the supported deep
learning libraries are installed. For more details, check Deep Learning Libraries Installer for ArcGIS.
Fine-tuning the model
This model cannot be fine-tuned using ArcGIS tools.
Input
8 bit, 3-band high-resolution (10-25 cm) imagery.
Output
Feature class containing separate masks for each tree.
Applicable geographies
The model is expected to work well in the United States.
Model architecture
This model is based upon the DeepForest python package which uses the RetinaNet model architecture implemented in torchvision and open-source Segment Anything Model (SAM) by Meta.
Accuracy metrics
This model has an precision score of 0.66 and recall of 0.79.
Training data
This model has been trained on NEON Tree Benchmark dataset, provided by the Weecology Lab at the University of Florida. The model also uses Segment Anything Model (SAM) by Meta that is trained on 1-Billion mask dataset (SA-1B) which comprises a diverse set of 11 million images and over 1 billion masks.
Sample results
Here are a few results from the model.
Citations
Weinstein, B.G.; Marconi, S.; Bohlman, S.; Zare, A.; White, E. Individual Tree-Crown Detection in RGB Imagery Using Semi-Supervised Deep Learning Neural Networks. Remote Sens. 2019, 11, 1309
Geographic Generalization in Airborne RGB Deep Learning Tree Detection Ben Weinstein, Sergio Marconi, Stephanie Bohlman, Alina Zare, Ethan P White bioRxiv 790071; doi: https://doi.org/10.1101/790071
An in-depth description of the item is not available.
Layers
Tools
Tables
Basemap
Project Contents:
Solution Contents
Contents
Layers
Screenshots
Terms of Use
No special restrictions or limitations on using the item's content have been provided.
Details
Dashboard views: Desktop
Source:
Creating data in:
Published as:
Other Views:
Dependent items in the recycle bin
Applicable: 2d
Size: 500.119 MB
ID: 6d910b29ff38406986da0abf1ce50836
Image Count: 0
Image Properties
Layer Drawing
Using tiles from a cache
Dynamically from data
Share
Owner
Folder
Categories
This item has not been categorized.
Tags
Segment anything, dlpk, LivingAtlasDLPK, tree, detection, Segmentation
Credits (Attribution)
No acknowledgements.Esri, Meta, Weecology research group at the University of Florida
Comments (27)
Can you fine tune/transfer learning this model with ArcPy?
I have the same problem as mattias.bovin. I am running ArcGIS Pro 3.2.2. Any solution?? A raster error has occurred. The messages that follow will provide more detail. ERROR 160117: The value type is incompatible with the field type. The value type is incompatible with the field type. [Confidence] [Failed to generate table] The value type is incompatible with the field type. [Confidence] Failed to execute (DetectObjectsUsingDeepLearning).
Hi, thank you again. I re-installed ArcGIS Pro and installed the Deep Learning Libraries installer again. I then tried to first run another deep learning package, the tree detection one: https://esrisverige.maps.arcgis.com/home/item.html?id=4af356858b1044908d9204f8b79ced99, and it works fine. However, when running this tree segmentation I still get the following error: Detect Objects Using Deep Learning ===================== Parameters Input Raster OrtofotoAvesta.tif Output Detected Objects C:\GIS\Lab\Lab.gdb\OrtofotoAvesta_DetectObjects9 Model Definition C:\GIS\ArcGIS\Packages\TreeSegmentation.dlpk Arguments padding 100;threshold 0,1;nms_overlap 0,1;batch_size 4;exclude_pad_detections True;test_time_augmentation False Non Maximum Suppression NO_NMS Confidence Score Field Confidence Class Value Field Class Max Overlap Ratio 0 Processing Mode PROCESS_AS_MOSAICKED_IMAGE Output Classified Raster ===================== Environments Processor Type GPU ===================== Messages Start Time: den 26 februari 2024 13:47:12 A raster error has occurred. The messages that follow will provide more detail. ERROR 160117: The value type is incompatible with the field type. [Failed to generate table] The value type is incompatible with the field type. [Confidence] Failed to execute (DetectObjectsUsingDeepLearning). Failed at den 26 februari 2024 13:48:34 (Elapsed Time: 1 minutes 21 seconds)
When running this package I end up with the following error messages, any ideas why this is happening? A raster error has occurred. The messages that follow will provide more detail. ERROR 160117: The value type is incompatible with the field type. [Failed to generate table] The value type is incompatible with the field type. [Confidence] Failed to execute (DetectObjectsUsingDeepLearning).
Hi, We tried to run the model on the sample data which you shared in ArcGIS 3.2.2 and we were able to successfully run it. There can be an issue with your environment. Can you try creating the environment using the Deep Learning Libraries Installer from this link https://github.com/Esri/deep-learning-frameworks. Can you also make sure that you have all the assets like the deep learning model and data on your local machine.
@ptuteja_geosaurus I have now shared the data in my content, thank you for having a look. I stumbled on the same error with another orthophoto though, so I'm not sure if the error depends on the data. Again, thanks for all the help!
@mattias.bovin Could you provide a sample of your data in the form of an arcgis item?
Hi, thank you for pointing that out! I still run into trouble I'm afraid: Detect Objects Using Deep Learning ===================== Parameters Input Raster OrtofotoAvesta.tif Output Detected Objects C:\GIS\Lab\Lab.gdb\OrtofotoAvesta_DetectObjects7 Model Definition C:\GIS\ArcGIS\Packages\TreeSegmentation.dlpk Arguments padding 100;threshold 0,1;nms_overlap 0,1;batch_size 4;exclude_pad_detections True;test_time_augmentation False;prompt box Non Maximum Suppression NO_NMS Confidence Score Field Confidence Class Value Field Class Max Overlap Ratio 0 Processing Mode PROCESS_AS_MOSAICKED_IMAGE Output Classified Raster ===================== Environments Processor Type GPU ===================== Messages Start Time: den 19 februari 2024 10:04:55 A raster error has occurred. The messages that follow will provide more detail. ERROR 160117: The value type is incompatible with the field type. [Failed to generate table] The value type is incompatible with the field type. [Confidence] Failed to execute (DetectObjectsUsingDeepLearning). Failed at den 19 februari 2024 10:06:06 (Elapsed Time: 1 minutes 11 seconds)
Hi, By looking at the error trace I can see that the input raster and model definition's name has spaces in it. Try to remove that and try again.
Sorry, the correct ArcGIS Python API version is 2.2.0.1.
Hi, thank you for helping out. I am using an orthophoto (RGB, 8-bit) with a resolution of 0.047 x 0.047 m taken from a drone. Using ArcGIS Pro 3.2.2, Python 3.9.18. Here is what I got: Detect Objects Using Deep Learning ===================== Parameters Input Raster Ortofoto AvestaX1 3maj202.tif Output Detected Objects C:\GIS\Lab\Lab.gdb\OrtofotoAvesta_DetectObjects1 Model Definition C:\Users\mabo\OneDrive - Esri Sverige AB\Documents\ArcGIS\Packages\TreeSegmentation.dlpk Arguments padding 100;threshold 0,1;nms_overlap 0,1;batch_size 4;exclude_pad_detections True;test_time_augmentation False;prompt box Non Maximum Suppression NO_NMS Confidence Score Field Confidence Class Value Field Class Max Overlap Ratio 0 Processing Mode PROCESS_AS_MOSAICKED_IMAGE Output Classified Raster ===================== Messages Start Time: den 13 februari 2024 14:14:34 A raster error has occurred. The messages that follow will provide more detail. ERROR 160117: The value type is incompatible with the field type. [Failed to generate table] The value type is incompatible with the field type. [Confidence] Failed to execute (DetectObjectsUsingDeepLearning). Failed at den 13 februari 2024 14:15:20 (Elapsed Time: 46,03 seconds)
Hi, Can you check if you are using the required Input Image and following the guide and if so can you also share the full error trace, the pro version and the arcgis python api version?
Hello, after running the model, I found that each tree is composed of circles smaller than itself. What is the reason for this?
Hi, This model takes the output bounding boxes from Tree Detection model ( https://arcg.is/0yLem5 ) and prompts SAM with the bounding box of the detected trees. SAM then produces segmentation masks for the trees that are converted to polygons and returned. You can try to use "center" as prompt in the model argument and check the results. Other than that you can also run the Tree Detection model and check the detections on your area.