The framework requires a foundation model that can segment images. We have experimented with LISA and SAM 3 for this task, and recommend using SAM 3. Please follow the setup instructions for SAM 3.
We use Gurobi for most of the mission planning. These steps are not required if you just want to use LAMP pipeline. Additionally, these instructions assume that you are able to get an academic license for Gurobi. This will require an academic email address and for you to either be connected to the internet through an academic network or be using a VPN that masks your IP address to match your academic institution. You will need to setup an account with Gurobi.
Go to the download center and download Gurobi Optimizer for your system. These instructions assume that you have a linux machine.
After downloading a copy of the solver, run these commands:
sudo cp gurobi<VERSION>_linux64.tar.gz /opt/
cd /opt/
sudo tar -xvzf gurobi<VERSION>_linux64.tar.gz
sudo rm gurobi<VERSION>_linux64.tar.gz
Add the following to your ~/.bashrc:
export GUROBI_HOME="/opt/gurobi<VERSION>/linux64"
export PATH="${PATH}:${GUROBI_HOME}/bin"
export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:${GUROBI_HOME}/lib"
Source .bashrc or open a new terminal.
source ~/.bashrc
Go to the Gurobi licenses portal and request a new license. This step requires you to be connected to the internet through an academic network. Once you generate a license, the website will give you a key and the next command:
grbgetkey <YOUR-LICENSE>
Build the C++ library:
cd $GUROBI_HOME/src/build
sudo make
sudo cp libgurobi_c++.a ../../lib/
You will need a C++ JSON reader for this project. We are using the nlohmann/json JSON reader:
sudo apt-get install nlohmann-json3-dev
To run LAMP, you will first want to configure your setup in scripts/defines.py. Be sure to update SAM_3_LOCATION and OPENAI_TOKEN (requires an OpenAI Token). Next, either generate a QGroundControl mission that has a take-off location and your desired geofences or capture an image of the environment.
With the SAM 3 environment that you setup while installing SAM, run scripts/heat_map_gen.py. The script will prompt you for your mission file (or sim image), will ask if you want to generate waypoints for this mission, and ask for the object of interest. The details of everything that the script generated will be saved in the META_DATA_FILE file, including the final mask and any generated mission files.
To generate the vorinoi cells over mission waypoints using the heat map from above, run the scripts/waypoint_scorer.py script. This will save scored waypoints at SCORED_WAYPOINT location and will create the vorinoi cell image in the directory where you ran the script.
The mission planner (which requires Gurobi) is a traditional C++ project that relies on cmake. First make sure that the linker is looking for the correct version of Gurobi. Open optimization_core/CMakeLists.txt and look for the linked libraries line:
target_link_libraries(optimization_core_lib gurobi_c++ gurobi130 m)
Update the gurobi130 to match your version of Gurobi. Next, create a build directory and run cmake:
mkdir optimization_core/build
cd optimization_core/build
cmake ..
make
If everything built without an issue, you should be able to run the solver:
./main