=========================== 1. Lincoln Field dataset =========================== The Ruth J. Simmons Quadrangle in Providence, RI, formerly known as Lincoln Field, is Brown University's historic lower campus, bound on four sides by all varieties of academic disciplines and student life. A multiple view image dataset was collected there throughout the year of 2006 to reveal daily, monthly and seasonal illumination changes. The dataset is useful to test outdoors computer vision algorithms such as multiple view reconstruction and 3-d change detection. ============================== 2. Dataset Brief Description ============================== Camera: the dataset was collected using a Nikon Coolpix 5000. Camera settings were not fixed, including white balance, shutter speed, exposure, aperture, ISO and focus. Many settings were set to AUTO mode. Acquisition times: pictures were taken roughly at 9:00 a.m., 1:00 p.m. and 5:00 p.m., three times a week from April/2006 until November/2006. Exact times can be extracted from Jpeg EXIF timestamps in each image file. Viewpoints: photos were taken with the photographer standing at roughly 6 different locations on the roofs of surrounding buildings at angles where most of the field was visible. From some viewpoints, multiple angles were necessary to capture the entire field. Acquisition: the camera was in photographer's hands (no tripods were used). Therefore, camera location and orientation always change at least slightly from a picture to another. Imagery content: background includes buildings, trees, ground, lampposts etc. Dynamic foreground objects include vehicles, people and their belongings, trash cans etc. Illumination: the illumination changes according to time and day (Sun position) and weather conditions (clouds, rain). Sunlight orientation can be computed from image timestamps and geographical coordinates. ====================== 3. Technical Details ====================== Lincoln Field coordinates: 41.82608° N, 71.39991° W (+41.82608, -71.39991). Providence time zone: (UTC-05:00) Eastern Time (US & Canada). Daylight saving time (DST) in 2006: Started 2 a.m. on April 2 and ended at 2 a.m. on October 29. DST in the dataset: if using image timestamps, DST must be considered. All image timestamps are under DST, i.e. equivalent to time zone (UTC-04:00). Note: ideally, only images until 10/29/2006 should have been under DST, but the camera time was not reverted back. Photographers: Osman Sezer acquired images until 06/09/2006. Eduardo Almeida acquired all other images (starting on 17/07/2006). ========================= 4. Filename conventions ========================= The filename convention is "- hm - .JPG", e.g. 20061016-17high 17h23m40 - center2.JPG, where: stands for 4 digit year followed by 2 digit month and 2 digit day of the date the picture was taken. ,, stand for 2 digit hour, 2 digit minute and 2 digit seconds, of the time the picture was taken. refers to the 9:00 a.m., 1:00 p.m. (13h00) or 5:00 p.m. (17h00) image acquisition times describe above, where is respectively 09, 13 or 17. Note that if, e.g., pictures from the 9:00 a.m. section were taken late at 10:05 a.m., will be still 09, while will be 10 and will be 05. refers to photographer location and can be {fire, metcalf, left, center, right}. The label `fire' refers to a fire escape on the Metcalf building, `metcalf' to the roof of the Metcalf building, `left', `center', `right' refer to positions within the wide roof of the Lincoln Field building (`left' and `right' are roof corners relative to a person standing at the `center' of the roof facing the grass field in front of it). The Lincoln Field building is seen from pictures taken from the Metcalf building. The Metcalf building is typically seen from pictures from the Lincoln Field building. Note the label provides good prior information about the camera center parameters. is a scalar enumerating multiple angles necessary to capture the entire field from a given viewpoint, as described in Section2:Viewpoints. If only one angle was used, is 1. Note this label defines where the camera points to, but still leaves 1 degree of freedom for the camera pose, the rotation about the optical axis. For instance, pictures from the fire escape viewpoint captured the entire field, so all those images are labeled `fire1', while 3 pictures were needed to capture the field from the `right' corner of the Lincoln Field Building, so those are labeled `right1', `right2' and `right3'. is a string. It can be {low, high}. `low' indicates a resolution of 1600 x 1200. `high' indicates the highest (native) camera resolution of 2560 x 1920. Notes: The labels , and were hand labeled. The labels , , , and are useful to filter images searches for a given date, time of the day, image resolution or camera pose. ======================= 5. Folder conventions ======================= The dataset is provided in two different flavors: 1. UNPROCESSED. 2. CALIBRATED. 1. UNPROCESSED --------------- This dataset category contains the original (unprocessed) JPEG files downloaded from the camera memory card. The default camera file names were renamed to follow the convention above. Images are distributed into many sub-directories named according to date and collection schedule. The folder names follow similar convention as the filenames. Folder names are "-" where and are defined as for the filenames and represent the date and collection section schedule of all the files inside the folder. is the string "high" (without quotes) for folder with filenames with resolution 2560 x 1920, otherwise it is an empty string. Total number of UNPROCESSED images: 3742 Number of images by label: fire: 462 metcalf: 818 right: 890 center: 743 left: 829 All files contain original JPEG EXIF tags. Images contain lens distortion. 2. CALIBRATED -------------- This dataset category contains a subset of the UNPROCESSED category images that have been processed to remove effects of lens distortion and includes estimated cameras. The file names did not changed, however files are now distributed in different sub-directories according to label. Estimated camera pose parameters files are included for each image. The camera file name and path are the same as the associated image file, and the extension is .txt. The camera file provides a 3x4 projection matrix P = K*[R T]. Total number of CALIBRATED images: 870 Total number of cameras: 870 Number of images by label: fire: 173 metcalf: 329 right: 78 center: 112 left: 178 Images contain no EXIF data, since it was lost during the undistortion process. All images were undistorted based on a single set of lens distortion parameters estimated from a small subset of images (~30) and their individual hand-marked correspondences with known measured 3-d points. For each viewpoint, parameters for one particular view were estimated using hand-marked image correspondences with known 3-d points. Relative camera poses for all other cameras in a viewpoint were estimated using planar homographies. The homographies were estimated automatically using Generalized Dual Bootstrap-ICP Algorithm [2] and the relative poses computed from characteristic lines [1]. Note: the only labels which have calibrated images are {fire1, metcalf1, metcalf2, right1, center1, left1}. ======================== 6. Measured 3-d points ======================== 3-d measurements are used for camera parameters estimation and/or georegistration. WORLD.TXT ---------- The plain text file world.txt has 102 3-d points measured at the scene using differential GPS. Points are coordinates of Euclidean (x,y,z) locations. Units are in feet. The coordinates format is unknown, but it is likely to be Rhode Island State Planes NAD83 (RI FIPS 3800 NAD83), such that (x,y,z) are respectively Easting, Northing and Elevation. The 3-d points were chosen at feature locations easily detected in images such as the top of lampposts, building corners, curve intersections on the concrete and center of manholes. Conversions to other coordinates (assuming RI FIPS 3800 NAD83), such as UTM and LAT/LON, are provided. A file containing labels of some interest points (mainly above the ground) is also provided. Among the labels, there are points associated with the = {fire, metcalf, left, center, right} file and folder label, measured at the neighborhood of where the photographer stood. The data in WORLD.TXT is also provided in an Excel file WORLD.XLS. WORLD2.TXT ----------- Additional points were measured with lower accuracy using measure tape (accuracies unknown). These additional points are provided in the WORLD2.TXT file. The file contains both the original points in WORLD.TXT and the extra ones. Moreover, a change of coordinate frame were applied such that ground points coordinates are roughly aligned to the x-y plane, the origin is close to the center of the field and z is altitude. These coordinates were not converted to other formats. These additional points are typically corners on the bottom of the windows of the buildings. ======================== 7. Interesting objects ======================== Two intentional background changes were introduced to create new case studies for research purposes: 1. Fake rock. 2. Hole in the ground. 2. The Rock -------------- A fake rock was placed in the center of the field to suggest appearance of new static object. The first pictures with the rock was taken on 09/06/2006 and the last on 09/22/2006. The rock was always stored in a secure location and at collection times, placed on the same predefined position based on markers on the ground. It was common to see sometimes curious people around the fake rock. 2. The Hole -------------- A hole was dug in the grass field to represent another category of change: a missing part of static background. The hole appears in the dataset starting on 08/23/2006 1:00 p.m. until 08/24/2006. It was common to see sometimes curious people around the hole. ================= 8. Ground Truth ================= For a given set of 24 images, hand-labeled masks highlighting surface appearance changes and new foreground objects are provided to evaluate change detection algorithms. The masks are 1 bit/pixel PNG images with the same name as their associated files, the ones in the undistorted CALIBRATED dataset. Shadows are only marked if due to a new foreground object. =============== 9. References =============== [1] J. Wang and Y. Liu, "Characteristic Line of Planar Homography Matrix and Its Applications in Camera Calibration." International Conference on Pattern Recognition (ICPR), vol. 1, pp. 147-150, 2006. [2] Gehua Yang, Charles V. Stewart, Michal Sofka, and Chia-Ling Tsai, "Registration of Challenging Image Pairs: Initialization, Estimation, and Decision" IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 11, pp. 1973-1989, Nov. 2007.