VizWiz Salient Object Detection using ConvNeXt and F3Net

by Conner Hnatiuk

Introduction

Included:

• src: The source code of the project
• scripts: Holds scripts for image and list management
• eval: Evaluation tools
  • maps: Where the saliency maps are placed
• data: Where the images and masks should be placed
• requirements.txt: The pip requirements

Prerequisites

Training / Testing

If pip is installed run

pip install -r requirements.txt

• Python >= 3.10.6
• Pytorch >= 2.0
• TorchVision >= 0.15
• OpenCV 4.0
• Numpy
• Timm >= 0.6.13
• Six
• TensorBoardX
• tqdm

Analysis

• Matlab

Dataset

Data

|data
    |VizWiz
        |VizWiz-TR
            |images: xxx.jpg ...
            |masks : xxx.png ...
            |train.txt
        |VizWiz-VA
            |images: xxx.jpg ...
            |masks : xxx.png ...
            |test.txt

The dataset can be found on the VizWiz Grand Challenge: Salient object detection page

Or individually here

Train || Validation || Test

Annotations

To obtain the masks ensure that the following folders exist, that the original dataset images are in their resepective image folder and that the annotations are in their respective folder

|data
    |VizWiz
        |VizWiz-TR
            |image
            |mask
            |VizWiz_SOD_train_challenge.json
        |VizWiz-VA
            |image
            |mask
            |VizWiz_SOD_val_challenge.json
        |VizWiz-TR-Sub
            |image
            |mask

Then do the following from the main root of the project

cd scripts
./get_masks.py <train|val>

To obtain the subset

cd scripts
./get_sub.py

My subset can be found here

VizWiz-TR-Sub

Password: VizSub

Pre-trained Model

The pre-trained model should download upon first time execution, if not it can be found here

Train

To train the model run the following

cd src
python3 train.py

If need be this command can be run from a notebook by runniny

!python3 train.py

Test

To test the model run the following

cd src
python3 test.py

If need be this command can be run from a notebook by runniny

!python3 test.py

Evaluation

To evaluate the model ensure that MATLAB is installed. This evaluation code is part of the F3Net Project

Use MATLAB to run main.m

cd eval
matlab
main

Authors and Acknowledgements

This project is based on the following implementations:

• F3Net
• ConvNeXt