NR2R (Night RAW to RGB)

To form the collection of nighttime RAW samples, we first selected a total of 150 images with the spatial resolution at 3464×5202 from the training and validation sets provided by the night image challenge. And then these RAW images are pre-processed to best produce noise-free samples using a notable CNN based denoiser. This is because nighttime imaging experiences a very challenging situation with heavy noises incurred by high ISO setting under poor illumination condition (e.g., underexposure).

We applied a two-stage process to derive the corresponding RGB image of each RAW input. We first used a simple ISP that was comprised of linear demosaicing, gray-world white balance, color correction, and gamma correction to convert each denoised RAW input to its RGB format for groundtruth illumination estimation. To this aim, we mark the “White Patch” from each converted RGB, where the patch is presented in neutral gray, and its RGB channels are approximately the same. Since the gray surface presumably reflects all incoming light radiation, it can be used to represent the ground truth illumination of the RAW image accordingly. We then perform the 2-stage labeling using the illumination from the 1-stage. Specifically, first we get the correct color image by a serial operations including linear demosaicing, white balance using the label white balance and color correction with the camera inner color correction matrix (CCM). The brightness adjustment consists of local and global tone mapping jointly. Since local tone mapping requires fine-grained adjustment of each small patch in the scene, it is difficult to annotate it manually. Therefore, we use a pre-trained local tone mapping model to fulfill the task. Since the pre-trained tone mapping network was trained using daytime image, it is good for local adjustment, but fails to control the global brightness. We save the model output using a 16-bit intermediate format in PNG, and then import it into the Lightroom app to adjust the global exposure, brightness, shadows and contrast manually for final high-quality RGB image rendering, with which we emulate the image rendering knowledge from Professional Photographers. Thereafter, we successfully obtain a high-resolution nighttime RAW-RGB image dataset.