Autonomous Vehicle Control
For my senior design project, we developed and integrated a complete autonomous driving pipeline in a Gazebo environment, using ROS2 as one of the main tools. This project begins with classical control methods such as PID and Pure Pursuit and extends the system to handle learning-based perception for dynamic motion planning, which is what the vehicle uses to navigate throughout the track in Gazebo.
Inspiration
My motivation for choosing this topic came from a personal experience I had while driving on the highway. I remember seeing a Tesla driving right next to me in Autopilot mode, maintaining its lane, adjusting its speed, and responding to the surrounding traffic without any input from the driver. I didn't jump into understanding the architecture behind AVs right away, but as I was exploring topics to work with for my senior design project, the idea of working with AVs came back to my mind, which left me with no options but to pick this topic.
What it does
The project consists of algorithms that operates the AV on a simulated track in Gazebo environment. We did not have a physical vehicle to test this out, that is where the idea of using Gazebo came from. We wanted to create an environment where we could test our algorithms in a simulated environment. We also used RViz to visualize the output of our algorithms. This project begins with classical control methods and extends the system to handle learning-based perception for dynamic motion planning, which is what the vehicle uses to navigate throughout the track in Gazebo
How I built it
We implemented the PID controller as a ROS2 node by extending the Node class from the Python ROS2 client library. The controller subscribed to the /waypoints topic to receive the target velocity and used data from the /odom topic to determine the vehicle’s current state. At each control update, we calculated the error between the target and current values and applied the proportional, integral, and derivative terms using tuned Kp, Ki, and Kd gains. ROS2 timestamps were used to calculate the time step for accurate integration and differentiation. The resulting control signal was clamped to prevent excessive actuation before being published as a velocity command through /cmd_vel. To evaluate controller performance, we also logged time-stamped speed, error, and control-output values to a CSV file, allowing us to analyze the vehicle’s behavior and tune the controller after each simulation run. This PID controller formed one part of the larger autonomous driving pipeline, working alongside Pure Pursuit for steering and, later, a CNN-based perception module that generated dynamic waypoints from camera images.
Challenges I ran into
The biggest challenge I faced during this project was training the U-Net-based convolutional neural network (CNN) for lane segmentation. Since I was working on my Mac, I did not have sufficient GPU resources to train the model and obtain the best epoch model with lower validation loss. Training a deep learning model can be computationally expensive, especially when working with image-based semantic segmentation, where each training sample contains a full image and corresponding pixel-level labels.
Accomplishments I'm proud of
As mentioned, the biggest challenge was to obtain the best epoch model. To overcome this challenge, I looked for ways to train the model on different servers. I used Google Colab to train the model. It was a tedious process to train the model and obtain the best epoch model. Instead of relying entirely on my Mac’s limited hardware, I uploaded the dataset and training pipeline to Colab and used its available GPU resources to significantly speed up model training. This allowed me to run more training experiments, compare different numbers of epochs, and evaluate the model’s performance without being constrained by my local hardware
What I learned
I learned about the importance of Docker and Containerization. Without Docker and the Gazebo image available, it would take us months to set up the environment and get the algorithms working. I also realized the role unit tests play when it comes to the implementation of complex software design. There were major errors we could have avoided if we had wrote tests and function implementation simultaneously
What's next
My goal is to integrate the algorithm in a dynamic environment with real-time traffic to evaluate multi-object detection, collect sensor data to continuously refine the model, and eventually test the pipeline on a physical AV. As of now, the vehicle can operate on an empty track with no objects around, navigate and make the turns based on lane curvature, however, the real challenge begins when we have to incorporate other AVs, pedestrians, and cyclists into the environment.
One thing I'd redo
If I were to start this project over, I would adopt a Waterfall model rather than Agile. When we started, we assumed we could quickly build a prototype and iteratively refine it. However, lacking requirements and specifications led to major integration issues mid-project. One example I can think of was adjusting the vehicle's velocity around sharp turns. Because curvature-dependent speed profiles weren't defined in the original controller spec, the AV struggled with overshooting on sharp bends. We also misunderstood the concept of calculating the lookahead distance, a major component that was used to control the logic and calculate the angular velocity for Pure Pursuit algorithm. Had we documented the specification properly, we would have been able to handle such cases much more efficiently