ML4Science 2023: Phase masks implementations

README.rst

@@ -1,70 +1,20 @@
 =============
 LenslessPiCam
 =============
+Ahmed Elalamy (324610), Seif Hamed (312081), Ghita Tagemouati (330383)
 
-.. image:: https://readthedocs.org/projects/lensless/badge/?version=latest
- :target: http://lensless.readthedocs.io/en/latest/
- :alt: Documentation Status
-
-
-.. image:: https://joss.theoj.org/papers/10.21105/joss.04747/status.svg
- :target: https://doi.org/10.21105/joss.04747
- :alt: DOI
-
-.. image:: https://static.pepy.tech/badge/lensless
- :target: https://www.pepy.tech/projects/lensless
- :alt: Downloads
-
-
-*A Hardware and Software Toolkit for Lensless Computational Imaging with a Raspberry Pi*
------------------------------------------------------------------------------------------
-
-.. image:: https://github.com/LCAV/LenslessPiCam/raw/main/scripts/recon/example.png
- :alt: Lensless imaging example
- :align: center
-
-
-This toolkit has everything you need to perform imaging with a lensless
-camera. We make use of a low-cost implementation of DiffuserCam [1]_, 
-where we use a piece of tape instead of the lens and the
-`Raspberry Pi HQ camera sensor <https://www.raspberrypi.com/products/raspberry-pi-high-quality-camera>`__
-(the `V2 sensor <https://www.raspberrypi.com/products/camera-module-v2/>`__
-is also supported). Similar principles and methods can be used for a
-different lensless encoder and a different sensor. 
-
-*If you are interested in exploring reconstruction algorithms without building the camera, that is entirely possible!*
-The provided reconstruction algorithms can be used with the provided data or simulated data.
-
-We've also written a few Medium articles to guide users through the process
-of building the camera, measuring data with it, and reconstruction.
-They are all laid out in `this post <https://medium.com/@bezzam/a-complete-lensless-imaging-tutorial-hardware-software-and-algorithms-8873fa81a660>`__.
+Our work is mainly done in the trainable_mask.py and mask.py files.
 
 Setup 
 -----
 
-If you are just interested in using the reconstruction algorithms and 
-plotting / evaluation tools you can install the package via ``pip``:
+First, install the lensless package
 
 .. code:: bash
 
  pip install lensless
 
 
-For plotting, you may also need to install
-`Tk <https://stackoverflow.com/questions/5459444/tkinter-python-may-not-be-configured-for-tk>`__.
-
-
-For performing measurements, the expected workflow is to have a local 
-computer which interfaces remotely with a Raspberry Pi equipped with 
-the HQ camera sensor (or V2 sensor). Instructions on building the camera
-can be found `here <https://lensless.readthedocs.io/en/latest/building.html>`__.
-
-The software from this repository has to be installed on **both** your
-local machine and the Raspberry Pi. Note that we highly recommend using
-Python 3.9, as some Python library versions may not be available with 
-earlier versions of Python. Moreover, its `end-of-life <https://endoflife.date/python>`__ 
-is Oct 2025.
-
 *Local machine setup*
 =====================
 
@@ -92,101 +42,8 @@ install the library locally.
 
  # extra dependencies for local machine for plotting/reconstruction
  pip install -r recon_requirements.txt
+ pip install -r mask_requirements.txt
 
- # (optional) try reconstruction on local machine
- python scripts/recon/admm.py
-
- # (optional) try reconstruction on local machine with GPU
- python scripts/recon/admm.py -cn pytorch
-
-
-Note (25-04-2023): for using the :py:class:`~lensless.recon.apgd.APGD` reconstruction method based on Pycsou
-(now `Pyxu <https://github.com/matthieumeo/pyxu>`__), a specific commit has 
-to be installed (as there was no release at the time of implementation):
-
-.. code:: bash
-
- pip install git+https://github.com/matthieumeo/pycsou.git@38e9929c29509d350a7ff12c514e2880fdc99d6e
-
-If PyTorch is installed, you will need to be sure to have PyTorch 2.0 or higher, 
-as Pycsou is not compatible with earlier versions of PyTorch. Moreover, 
-Pycsou requires Python within 
-`[3.9, 3.11) <https://github.com/matthieumeo/pycsou/blob/v2-dev/setup.cfg#L28>`__.
-
-Moreover, ``numba`` (requirement for Pycsou V2) may require an older version of NumPy:
-
-.. code:: bash
-
- pip install numpy==1.23.5
-
-*Raspberry Pi setup*
-====================
-
-After `flashing your Raspberry Pi with SSH enabled <https://medium.com/@bezzam/setting-up-a-raspberry-pi-without-a-monitor-headless-9a3c2337f329>`__, 
-you need to set it up for `passwordless access <https://medium.com/@bezzam/headless-and-passwordless-interfacing-with-a-raspberry-pi-ssh-453dd75154c3>`__. 
-Do not set a password for your SSH key pair, as this will not work with the
-provided scripts.
-
-On the Raspberry Pi, you can then run the following commands (from the ``home`` 
-directory):
-
-.. code:: bash
-
- # dependencies
- sudo apt-get install -y libimage-exiftool-perl libatlas-base-dev \
- python3-numpy python3-scipy python3-opencv
- sudo pip3 install -U virtualenv
-
- # download from GitHub
- git clone [email protected]:LCAV/LenslessPiCam.git
-
- # install in virtual environment
- cd LenslessPiCam
- virtualenv --system-site-packages -p python3 lensless_env
- source lensless_env/bin/activate
- pip install --no-deps -e .
- pip install -r rpi_requirements.txt
-
-
-Acknowledgements
-----------------
-
-The idea of building a lensless camera from a Raspberry Pi and a piece of 
-tape comes from Prof. Laura Waller's group at UC Berkeley. So a huge kudos 
-to them for the idea and making tools/code/data available! Below is some of 
-the work that has inspired this toolkit:
-
-* `Build your own DiffuserCam tutorial <https://waller-lab.github.io/DiffuserCam/tutorial>`__.
-* `DiffuserCam Lensless MIR Flickr dataset <https://waller-lab.github.io/LenslessLearning/dataset.html>`__ [2]_. 
-
-A few students at EPFL have also contributed to this project:
-
-* Julien Sahli: support and extension of algorithms for 3D.
-* Yohann Perron: unrolled algorithms for reconstruction.
-
-Citing this work
-----------------
-
-If you use these tools in your own research, please cite the following:
-
-::
-
- @article{Bezzam2023,
- doi = {10.21105/joss.04747},
- url = {https://doi.org/10.21105/joss.04747},
- year = {2023},
- publisher = {The Open Journal},
- volume = {8},
- number = {86},
- pages = {4747},
- author = {Eric Bezzam and Sepand Kashani and Martin Vetterli and Matthieu Simeoni},
- title = {LenslessPiCam: A Hardware and Software Platform for Lensless Computational Imaging with a Raspberry Pi},
- journal = {Journal of Open Source Software}
- }
-
-References
-----------
-
-.. [1] Antipa, N., Kuo, G., Heckel, R., Mildenhall, B., Bostan, E., Ng, R., & Waller, L. (2018). DiffuserCam: lensless single-exposure 3D imaging. Optica, 5(1), 1-9.
+ # training with the height varying mask
+ python scripts/recon/train_unrolled.py -cn train_heightvarying
 
-.. [2] Monakhova, K., Yurtsever, J., Kuo, G., Antipa, N., Yanny, K., & Waller, L. (2019). Learned reconstructions for practical mask-based lensless imaging. Optics express, 27(20), 28075-28090.

configs/train_coded_aperture.yaml

@@ -0,0 +1,55 @@
+# python scripts/recon/train_unrolled.py -cn train_coded_aperture
+defaults:
+ - train_unrolledADMM
+ - _self_
+
+# Train Dataset
+files:
+ dataset: mnist # Simulated : "mnist", "fashion_mnist", "cifar10", "CelebA". Measure :"DiffuserCam"
+ celeba_root: /scratch/bezzam
+ downsample: 16 # TODO use downsample simulation instead? 
+ n_files: 100
+ crop:
+ vertical: [810, 2240]
+ horizontal: [1310, 2750]
+
+torch_device: "cuda" 
+
+optimizer:
+ # type: Adam # Adam, SGD...
+ # lr: 1e-4
+ type: SGD
+ lr: 0.01
+
+#Trainable Mask
+trainable_mask:
+ mask_type: TrainableCodedAperture
+ # optimizer: Adam
+ # mask_lr: 1e-3
+ optimizer: SGD
+ mask_lr: 0.01
+ L1_strength: False
+ binary: False
+ initial_value:
+ psf_wavelength: [550e-9]
+ method: MLS
+ n_bits: 8 # (2**n_bits-1, 2**n_bits-1)
+ # method: MURA
+ # n_bits: 25 # (4*nbits*1, 4*nbits*1)
+ # # -- applicable for phase masks
+ # design_wv: 550e-9 
+
+simulation:
+ grayscale: True
+ flip: False
+ scene2mask: 40e-2
+ mask2sensor: 2e-3
+ sensor: "rpi_hq"
+ object_height: 0.30
+
+training:
+ crop_preloss: True # crop region for computing loss
+ batch_size: 4
+ epoch: 25
+ eval_batch_size: 16
+ save_every: 1

configs/train_heightvarying.yaml

@@ -0,0 +1,43 @@
+# python scripts/recon/train_unrolled.py -cn train_multilens_array
+defaults:
+ - train_unrolledADMM
+ - _self_
+
+# Train Dataset
+files:
+ dataset: fashion_mnist # Simulated : "mnist", "fashion_mnist", "cifar10", "CelebA". Measure :"DiffuserCam"
+ celeba_root: /scratch/bezzam
+ downsample: 16 # TODO use simulation instead? 
+ n_files: 100
+ crop:
+ vertical: [810, 2240]
+ horizontal: [1310, 2750]
+
+torch_device: "cuda:0" 
+
+#Trainable Mask
+trainable_mask:
+ mask_type: TrainableHeightVarying
+ optimizer: Adam
+ mask_lr: 1e-3
+ L1_strength: False
+ binary: False
+ initial_value: 
+ psf_wavelength: [550e-9]
+ design_wv: 550e-9
+
+simulation:
+ grayscale: True
+ flip: False
+ scene2mask: 40e-2
+ mask2sensor: 2e-3
+ sensor: "rpi_hq"
+ downsample: 16
+ object_height: 0.30
+
+training:
+ crop_preloss: True # crop region for computing loss
+ batch_size: 2
+ epoch: 25
+ eval_batch_size: 16
+ save_every: 1

configs/train_multilens_array.yaml

@@ -0,0 +1,44 @@
+# python scripts/recon/train_unrolled.py -cn train_multilens_array
+defaults:
+ - train_unrolledADMM
+ - _self_
+
+# Train Dataset
+files:
+ dataset: fashion_mnist # Simulated : "mnist", "fashion_mnist", "cifar10", "CelebA". Measure :"DiffuserCam"
+ celeba_root: /scratch/bezzam
+ downsample: 16 # TODO use simulation instead? 
+ n_files: 100
+ crop:
+ vertical: [810, 2240]
+ horizontal: [1310, 2750]
+
+torch_device: "cpu" 
+
+#Trainable Mask
+trainable_mask:
+ mask_type: TrainableMultiLensArray
+ optimizer: Adam
+ mask_lr: 1e-3
+ L1_strength: False
+ binary: False
+ initial_value:
+ N : 10 #TODO: check this value ? 
+ psf_wavelength: [550e-9]
+ design_wv: 550e-9
+
+simulation:
+ grayscale: True
+ flip: False
+ scene2mask: 40e-2
+ mask2sensor: 2e-3
+ sensor: "rpi_hq"
+ downsample: 16
+ object_height: 0.30
+
+training:
+ crop_preloss: True # crop region for computing loss
+ batch_size: 2
+ epoch: 25
+ eval_batch_size: 16
+ save_every: 1

configs/train_unrolledADMM.yaml

@@ -84,6 +84,7 @@ trainable_mask:
  initial_value: psf
  grayscale: False
  mask_lr: 1e-3
+ optimizer: Adam # Adam, SGD... (Pytorch class)
  L1_strength: 1.0 #False or float
 
 target: "object_plane" # "original" or "object_plane" or "label"
@@ -129,7 +130,7 @@ training:
  crop_preloss: True # crop region for computing loss
 
 optimizer:
- type: Adam
+ type: Adam # Adam, SGD... (Pytorch class)
  lr: 1e-4
  slow_start: False #float how much to reduce lr for first epoch
  # Decay LR in step fashion: https://pytorch.org/docs/stable/generated/torch.optim.lr_scheduler.StepLR.html

lensless/eval/benchmark.py

@@ -124,8 +124,8 @@ def benchmark(
 
  for i, idx in enumerate(batch_idx):
  if idx in save_idx:
- prediction_np = prediction.cpu().numpy()[i].squeeze()
- # switch to [H, W, C]
+ prediction_np = prediction.cpu().numpy()[i]
+ # switch to [H, W, C] for saving
  prediction_np = np.moveaxis(prediction_np, 0, -1)
  save_image(prediction_np, fp=os.path.join(output_dir, f"{idx}.png"))