where even minor perception errors can cause instability or accidents. Fiducial markers, commonly used in robotics and augmented reality for localization and pose estimation, are particularly valuable for enabling autonomous takeoff and landing of rotorcraft.

which fail to capture the complexity of real outdoor environments. Factors such as changing illumination, temperature, wind, and rain can significantly degrade marker detection and pose-estimation performance from flying vehicles.

bridging the gap between laboratory evaluations and real-world outdoor operations for autonomous aerial systems.

As a UAV approaches a landing pad, it must detect markers across multiple distance ranges—from far approach to close touchdown. To ensure reliable detection throughout the descent, we used a nested multi-scale fiducial marker layout, allowing consistent recognition at long, medium, and short ranges.

Marker detection was performed using the AprilTag library, which identifies the pixel coordinates of each marker’s corners. The pose estimation process then solved a Perspective-n-Point (PnP) problem, relating the known 3D coordinates of the marker corners to their detected 2D image projections to compute the UAV’s position and orientation relative to the landing pad.

To accurately perceive the fiducial markers, it was essential to synchronize the color and thermal camera inputs. I implemented a synchronization setup between the DJI Matrice 300 RTK’s onboard color camera and an added Boson 640 thermal camera, achieving a timing offset of less than 1 ms. The synchronization was carried out in two stages: each time the main camera captured a frame, it generated an electrical trigger signal, which the thermal camera received to capture its corresponding frame simultaneously. This ensured precise frame alignment across both visual and thermal modalities for reliable marker detection under varying environmental conditions.

I enabled the telemetry mode of the FLIR Boson 640 to extract embedded metadata directly from each frame. The telemetry data—appended to the last two rows of every 16-bit image—contains information such as sensor status and precise frame completion timestamps. Using a custom parser, I extracted the timestamp from the four 16-bit telemetry words, reconstructing a 64-bit millisecond counter representing the exact moment the camera finished capturing the frame.

On the host computer, each frame arrival was also recorded using the ROS system time. By comparing the Boson’s internal timestamps with the ROS timestamps, I calculated the communication offset between the camera and the onboard computer. This alignment allowed me to achieve precise temporal synchronization between the thermal camera and other onboard sensors, enabling accurate multi-sensor data fusion.

<!-- wp:code -->
<pre class="wp-block-code"><code>static std::optional&lt;uint64_t&gt; getBosonTimestampMs(const cv::Mat&amp; thermal_img) {
  if (thermal_img.empty() || thermal_img.type() != CV_16UC1 || thermal_img.rows &lt; 2) return std::nullopt;

  // Telemetry region (vendor/mode-specific): last 2 rows, flattened as uint16_t
  const cv::Mat tele = thermal_img.rowRange(thermal_img.rows - 2, thermal_img.rows);
  const auto* w = tele.ptr&lt;uint16_t&gt;(0);
  const size_t words = tele.total();

  // Example indices (verify for your Boson telemetry layout):
  if (words &lt;= 143) return std::nullopt;
  const uint16_t w0 = w&#91;140], w1 = w&#91;141], w2 = w&#91;142], w3 = w&#91;143];

  // Compose 64-bit ms counter from four 16-bit words (adjust if your device uses 32-bit)
  const uint64_t ts =
      (static_cast&lt;uint64_t&gt;(w0) &lt;&lt; 48) |
      (static_cast&lt;uint64_t&gt;(w1) &lt;&lt; 32) |
      (static_cast&lt;uint64_t&gt;(w2) &lt;&lt; 16) |
       static_cast&lt;uint64_t&gt;(w3);
  return ts;
}</code></pre>
<!-- /wp:code -->

static void publishThermal(
    ros::Publisher& img_pub,
    ros::Publisher& info_pub,
    const sensor_msgs::CameraInfo& ci_template,
    const cv::Mat& thermal_img,
    bool raw16_encoding,                // true: mono16 (RAW16), false: mono8 (AGC8)
    const std::string& frame_id)
{
  // Derive stamp from telemetry when available
  ros::Time stamp = ros::Time::now();
  if (auto ts = getBosonTimestampMs(thermal_img)) {
    const uint64_t ms = *ts;
    stamp = ros::Time(ms / 1000ULL, (ms % 1000ULL) * 1000000ULL);
  }

  // Build Image message
  sensor_msgs::Image img;
  img.header.stamp = stamp;
  img.header.frame_id = frame_id;
  img.height = thermal_img.rows;
  img.width  = thermal_img.cols;
  img.encoding = raw16_encoding ? "mono16" : "mono8";
  img.is_bigendian = false;                              // Boson outputs little-endian
  img.step = static_cast<uint32_t>(thermal_img.step);
  img.data.assign(thermal_img.datastart, thermal_img.dataend);
  img_pub.publish(img);

  // Publish matching CameraInfo
  sensor_msgs::CameraInfo ci = ci_template;
  ci.header.stamp = stamp;
  ci.header.frame_id = frame_id;
  info_pub.publish(ci);
}

Using the extracted telemetry timestamps, I integrated the data into the ROS publishing pipeline. Since ROS time follows the format sec.nsec, I converted the Boson’s millisecond-based timestamp into seconds and nanoseconds by dividing and multiplying by 1000, respectively. This ensured that each thermal frame carried a precise hardware-captured timestamp rather than relying solely on system time. The aligned timestamps were then published through ROS using both Image and CameraInfo topics, enabling accurate multi-sensor synchronization and consistent temporal reference across the system.

My initial goal was to synchronize the thermal camera’s frame rate with the UAV’s trigger signal. However, despite extensive testing, the thermal camera would not align with the drone’s 60 Hz frame rate and remained stuck around 23 Hz. After exhausting local debugging, I reached out to the camera’s OEM, the equipment provider, and other labs at UIUC that had used the same model. Through these conversations, I learned that the issue required a software-level adjustment—specifically switching the camera’s internal “Trigger Mode” to Master Mode, a setting poorly documented in the manuals. This experience taught me the value of persistence, clear communication, and collaboration when solving hardware-software integration problems.

This project was a turning point in my understanding of robotics. As an undergraduate mechanical engineering student, I had long sought hands-on experience beyond theory. Working with UAVs and integrating sensors through ROS gave me practical exposure to synchronization, real-time data processing, and embedded programming. These experiences not only strengthened my low-level coding skills but also laid the foundation for my later robotics projects, where I continued to build on what I learned about system integration and software-hardware interaction.