up

content/code/image_video_processing/basicvideotools/_index.md

@@ -2,10 +2,12 @@
 title: "BasicVideoTools"
 img: "mov.webp"
 image_alt: "BasicVideoTools Image"
-link: "./subpages/basic_video.html"
+link: "./basicvideotools/content"
 description: |
  A Matlab Toolbox with convenient functions to handle video data. It includes routines to read VQEG and LIVE databases, generate synthetic sequences with controlled 2D and 3D speed, spatio-temporal Fourier transforms, perceptual sensors and filters (V1 and MT cells), and spatio-temporal CSFs.
 references:
  - "Importance of quantiser design compared to optimal multigrid motion estimation in video coding. Malo, J., Ferri, F.J., Gutierrez, J., and Epifanio, I. Electronics Letters, 36(9):807-809, 2000."
  - "Video quality measures based on the standard spatial observer. Watson, A.B., and Malo, J. ICIP, 2002."
+type: "code"
+layout: "single"
 ---

content/code/image_video_processing/basicvideotools/content.md

@@ -0,0 +1,56 @@
+---
+title: "Basic Video Tools: A Matlab Toolbox for Video Data and Spatio-Temporal Vision Models (J. Malo, J. Gutirrez and V. Laparra (c) Universitat de Valncia 1996 - 2014)"
+abstract: |
+ # What is in BasicVideoTools?
+ BasicVideoTools is a Matlab/Octave Toolbox intendend to deal with video data and spatio-temporal vision models. In particular, it includes convenient *.m files to:
+ - Read standard (VQEG and LIVE) video data
+ - Rearrange video data (as for instance to perform statistical analysis)
+ - Generate controlled sequences (controlled contrast, texture, and 2d and 3d speed)
+ - Compute 3D Fourier transforms
+ - Play with motion perception models (spatial texture and motion-sensitive cells of LGN, V1 and MT, and spatio-temporal CSF)
+ - Visualize movies (achromatic only)
+
+ # What is not in BasicVideoTools?
+ 
+ BasicVideoTools does not include:
+ - Optical flow or motion estimation/compensation algorithms 
+ - Video Coding algorithms
+ - Video Quality Mesures
+ 
+ If you are looking for the above, please consider downloading other Toolboxes:
+ 
+ - Motion estimation:
+ - Video_coding.html ([Hierarchical Block Matching](http://www.scholarpedia.org/article/Optic_flow))
+ - [Video Coding (improved MPEG)](./../../videocodingtools/content) 
+ - [Video Quality](./../../videoqualitytools/content) 
+
+ # Download BasicVideoTools!
+
+ - [The code](https://huggingface.co/datasets/isp-uv-es/Web_site_legacy/resolve/main/code/soft_imvideo/vista_toolbox/BasicVideoTools_code.zip) (version 1.0. Use this version only for compatibility with the code in the experiments of the motion-aftereffect paper).
+ 
+ - [The code](https://huggingface.co/datasets/isp-uv-es/Web_site_legacy/resolve/main/code/soft_imvideo/basic_video/BasicVideoTools_v3.zip) (version 3.0 -Not only improved sampling functions and additional motion sensitive cells, but also more things)
+ 
+ - Optional data (not necessary to run the code): If you use these data please cite the VQEG and LIVE databases (for video), and the CVC Barcelona Database (for images)
+ 
+ - [Image data](https://huggingface.co/datasets/isp-uv-es/Web_site_legacy/resolve/main/code/soft_imvideo/basic_video/image_data.zip) (1.8 GB). Luminance images from the CVC Barcelona Calibrated Image Database.
+ 
+ - [Video data](https://huggingface.co/datasets/isp-uv-es/Web_site_legacy/resolve/main/code/soft_imvideo/basic_video/video_data.rar) (2.6 GB): Raw videos from the VQEG and LIVE video databases.
+
+ # Installation and Requirements
+
+ - Download the BasicVideoTools file(s)
+ - Decompress at your machine in the folder BasicVideoTools (no location restrictions for this folder)
+ - Update the matlab/octave path including all subfolders
+ - Tested on Matlab 2006b and posterior Matlab versions
+ 
+ * Video and image data are only required if you want to gather statistics from natural videos or from natural images with controlled speed
+
+ # How to get started?
+ For a general overview please take a look at the contents.m file, or (after you included it in th path) just look for help by typing the name of the folder, for instance: help BasicVideoTools_v2.
+ 
+ For additional details on how to use the functions in practice, see the demos:
+
+ - **demo_motion_programs**, demo on how to use most functions (except random dots and newtonian sequences).
+ - **example_random_dots_sequence**, demo on random dots sequences with controlled flow.
+ - **example_newtonian_sequence**, demo on physics-controlled sequences.
+---

content/code/image_video_processing/spatiospectraltools/_index.md

@@ -2,7 +2,7 @@
 title: "SpatioSpectralTools"
 img: "constancy.webp"
 image_alt: "SpatioSpectralTools Image"
-link: "./subpages/spatiospectral.html"
+link: "./spatiospectraltools/content"
 description: |
  SpatioSpectralTools is a Matlab Toolbox for reflectance and illuminant estimation that uses spatial information to simplify the (otherwise ill-conditioned) inverse problem. The proposed analysis is useful to derive the spatio-spectral resolution required to solve a retrieval problem.
 references:

content/code/image_video_processing/spatiospectraltools/content.md

@@ -0,0 +1,68 @@
+---
+title: "The role of spatial information in disentangling the irradiance-reflectance-transmitance ambiguity"
+abstract: |
+ In the satellite hyperspectral measures the contributions of light, surface, and atmosphere are mixed. Applications need separate access to the sources. Conventional inversion techniques usually take a pixel-wise, spectral-only approach. However, recent improvements in retrieving surface and atmosphere characteristics use heuristic spatial smoothness constraints.
+ 
+ In this paper we theoretically justify such heuristics by analyzing the impact of spatial information on the uncertainty of the solution. The proposed analysis allows to assess in advance the uniqueness (or robustness) of the solution depending on the curvature of a likelihood surface. In situations where pixel-based approaches become unreliable it turns out that the consideration of spatial information always makes the problem to be better conditioned. With the proposed analysis this is easily understood since the curvature is consistent with the complexity of the sources measured in terms of the number of significant eigenvalues (or free parameters in the problem). In agreement with recent results in hyperspectral image coding, spatial correlations in the sources imply that the intrinsic complexity of the spatio-spectral representation of the signal is always lower than its spectral-only counterpart. According to this, the number of free parameters in the spatio-spectral inverse problem is smaller so the spatio-spectral approaches are always better than spectral-only approaches.
+ 
+ Experiments using ensembles of actual reflectance values and realistic MODTRAN irradiance and atmosphere radiance and transmittance values show that the proposed analysis successfully predicts the practical difficulty of the problem and the improved quality of spatio-spectral retrieval.
+
+ ## Supplementary Material
+ 1. Extends the results in the manuscript to different spatial structures.
+ 2. Extends the results in the manuscript to different wavelength ranges and spatio-spectral resolutions.
+ 3. Statistically justifies the initialization scheme of sources.
+ 4. Provides sample data and code.
+
+ The generality of the conclusion is not surprising since the imaging equation and the PCA decompositions do not depend on the specific spatio-spectral resolution or wavelength range. The joint spatio-spectral approach will simplify the problem whenever there are relations between the signal at different spatial positions, which is true in a wide range of situations given the spatial continuity of the physical sources (the reflecting objects and the atmospheric phenomena).
+
+ ## 1. Effect of the Spatial Structure
+ Original and estimated reflectance images for sites of different spatial complexity (urban, forest, and open fields) using spectral-only and spatio-spectral retrieval. In these cases, the spatial resolution and wavelength range were the same as in the manuscript. No additional training was necessary, only the application of the previous analysis on new test locations.
+
+ ## 2. Effect of Wavelength Range and Spatio-Spectral Resolution
+ In this experiment, we used substantially different wavelength ranges and spatio-spectral resolutions from those in the manuscript.
+
+ ## 3. Initialization of the Sources (Surface Reflectance, Atmosphere Radiance, and Transmittance)
+ The retrieval procedure used to check the accuracy of the theoretical predictions involves a series of search loops that require an initialization of the variables S, A, and T. Since we searched in the decorrelated PCA domains, each coefficient of these sources was independently initialized using a random value drawn from the empirical marginal PDFs (histograms) learned at the training stage. Below we show examples of the marginal PDFs for some AC coefficients of these sources. The strong peak at zero makes zero initialization reasonable as well.
+
+
+referencias:
+ - nombre: "The role of spatial information in disentangling the irradiance-reflectance-transmitance ambiguity"
+ autores: "Sandra Jiménez and Jesús Malo"
+ publicacion: "Accepted to IEEE Trans. Geosci. Rem. Sens."
+ url: "https://huggingface.co/datasets/isp-uv-es/Web_site_legacy/resolve/main/code/soft_imvideo/spatiospectral/manuscr_TGRS_2012_00431.pdf"
+
+enlaces:
+ - nombre: "Download Data and Code"
+ url: "https://huggingface.co/datasets/isp-uv-es/Web_site_legacy/resolve/main/code/soft_imvideo/spatiospectral/code_retrieval_and_data.zip"
+ descripcion: "(*.zip file with data and Matlab toolbox. For running the examples, read and execute the demo_retrieval.m file)."
+
+imagenes:
+ - ruta: "org_contex.webp"
+ titulo: "[Original and Estimated Reflectance - Urban Context](#1-effect-of-the-spatial-structure)"
+ descripcion: "Reflectance images for urban, forest, and open field sites using spectral-only and spatio-spectral retrieval approaches."
+
+ - ruta: "reco_contex.webp"
+ titulo: "[Reconstructed Reflectance for Different Contexts](#1-effect-of-the-spatial-structure)"
+ descripcion: "Reconstructed reflectance images from different contexts, highlighting the difference between spectral-only and spatio-spectral retrieval."
+
+ - ruta: "errores.webp"
+ titulo: "[Error Maps for Reflectance Estimation](#2-effect-of-wavelength-range-and-spatio-spectral-resolution)"
+ descripcion: "Error maps showing the differences in reflectance estimation using spectral-only and spatio-spectral methods."
+
+ - ruta: "IR_example.webp"
+ titulo: "[Infrared Example - Different Wavelengths and Resolutions](#2-effect-of-wavelength-range-and-spatio-spectral-resolution)"
+ descripcion: "Example using different wavelength ranges and spatio-spectral resolutions in reflectance retrieval."
+
+ - ruta: "marginales_S.webp"
+ titulo: "[Marginal Distributions for Surface Reflectance](#3-initialization-of-the-sources-surface-reflectance-atmosphere-radiance-and-transmittance)"
+ descripcion: "Marginal PDFs for surface reflectance coefficients in the PCA domain."
+
+ - ruta: "marginales_A.webp"
+ titulo: "[Marginal Distributions for Atmosphere Radiance](#3-initialization-of-the-sources-surface-reflectance-atmosphere-radiance-and-transmittance)"
+ descripcion: "Marginal PDFs for atmospheric radiance coefficients in the PCA domain."
+
+ - ruta: "marginales_T.webp"
+ titulo: "[Marginal Distributions for Atmosphere Transmittance](#3-initialization-of-the-sources-surface-reflectance-atmosphere-radiance-and-transmittance)"
+ descripcion: "Marginal PDFs for atmospheric transmittance coefficients in the PCA domain."
+---
+

content/code/image_video_processing/videocodingtools/_index.md

@@ -2,7 +2,7 @@
 title: "VideoCodingTools"
 img: "cubo.webp"
 image_alt: "VideoCodingTools Image"
-link: "../soft_visioncolor/subpages/video_coding.html"
+link: "./videocodingtools/content"
 description: |
  VideoCodingTools is a Matlab Toolbox for motion estimation/compensation and video compression. Optical flow computation is done with perceptually meaningful hierarchical block matching, and residual quantization is done according to non-linear Human Visual System models.
 references:

content/code/image_video_processing/videocodingtools/content.md

@@ -0,0 +1,52 @@
+---
+title: "Motion Estimation and Video Coding Toolbox"
+abstract: |
+ # Motion Estimation
+ Our approach to motion estimation in video sequences was motivated by the general scheme of the current video coders with motion compensation (such as MPEG-X or H.26X [Musmann85, LeGall91, Tekalp95]).
+ 
+ In motion compensation video coders the input sequence, **A(t)**, is analized by a motion estimation system, M, that computes some description of the motion in the scene: typically the optical flow, **DVF(t)**. In the motion compensation module, **P**, this motion information can be used to predict the current frame, **A(t)**, from previous frames, **A(t-1)**. As the prediction, **(t)**, is not perfect, additional information is needed to reconstruct the sequence: the prediction error **DFD(t)**. This scheme is useful for video compression because the entropy of these two sources (motion, DVF, and errors, **DFD**) is significantly smaller than the entropy of the original sequence **A(t)**.
+ 
+ The coding gain can be even bigger if the error sequence is analyzed, and quantized, in an appropriate transform domain, as done in image compression procedures, using the transform **T** and the quantizer **Q**.
+
+ Conventional optical flow techniques (based in local maximization of the correlation by block matching) provide a motion description that may be redundant for a human viewer. Computational effort may be wasted describing 'perceptually irrelevant motions'. This inefficient behavior may also give rise to false alarms and noisy flows. To solve this problem, hierarchical optical flow techniques have been proposed (as for instance in MPEG-4 and in H.263). They start from a low resolution motion estimate and new motion information is locally added only in certain regions. However, new motion information should be added only if it is 'perceptually relevant'. Our contribution in motion estimation is a definition of 'perceptually relevant motion information' [Malo98, Malo01a, Malo01b]. This definition is based on the entropy of the image representation in the human cortex (Watson JOSA 87, Daugman IEEE T.Biom.Eng. 89): an increment in motion information is perceptually relevant if it contributes to decrease the entropy of the cortex representation of the prediction error. Numerical experiments (optical flow computation and flow-based segmentation) show that applying this definition to a particular hierarchical motion estimation algorithm, more robust and meaningful flows are obtained [Malo00b, Malo01a, Malo01b].
+ 
+ # Video Coding
+ As stated in the above scheme, the basic ingredients of motion compensation video coders are the motion estimation module, M, and the transform and quantization module, **T+Q**. Given our work in motion estimation and in image representation for efficient quantization, the improvement of the current video coding standards is straightforward. See [Malo01b] for a comprehensive review, and [Malo97b, Malo00a] for the original formulation and specific analysis of the relative relevance of M and **T+Q** in the video coding process.
+ 
+ Here is an example [Malo00a, Malo01b] of the relative gain in the reconstructed sequence (0.27 bits/pix) obtained from isolated improvements in motion estimation (**M**) and/or image representation and quantization (**T+Q**).
+ 
+ In the above distortion-per-frame plot, thick lines correspond to algorithms with poor (linear) quantization schemes and thin lines correspond to improved (non-linear) quantization schemes. Dashed lines correspond to algorithms with improved motion estimation schemes. The conclusion is that at the current bit rates an appropriate image representation and quantization is quite more important than improvements in motion estimation.
+
+referencias:
+ - nombre: "Perceptually weighted optical flow for motion-based segmentation in MPEG-4 paradigm"
+ autores: "J. Malo, J. Gutierrez, I. Epifanio, F. Ferri"
+ publicacion: "Electronics Letters, Vol. 36, 20, pp. 1693-1694 (2000)"
+ url: "https://huggingface.co/datasets/isp-uv-es/Web_site_legacy/resolve/main/code/soft_visioncolor/seg_ade2.ps"
+
+ - nombre: "Perceptual feed-back in multigrid motion estimation using an improved DCT quantization"
+ autores: "J. Malo, J. Gutierrez, I. Epifanio, F. Ferri, J.M. Artigas"
+ publicacion: "IEEE Transactions on Image Processing, Vol. 10, 10, pp. 1411-1427 (2001)"
+ url: "https://huggingface.co/datasets/isp-uv-es/Web_site_legacy/resolve/main/code/soft_visioncolor/ieeeoct01.pdf"
+
+ - nombre: "Importance of quantizer design compared to optimal multigrid motion estimation in video coding"
+ autores: "J. Malo, F. Ferri, J. Gutierrez, I. Epifanio"
+ publicacion: "Electronics Letters, Vol. 36, 9, pp. 807-809 (2000)"
+ url: "https://huggingface.co/datasets/isp-uv-es/Web_site_legacy/resolve/main/code/soft_visioncolor/elect00.ps"
+
+enlaces:
+ - nombre: "Motion_estimation_and_Video coding_code.zip"
+ url: "https://huggingface.co/datasets/isp-uv-es/Web_site_legacy/resolve/main/code/soft_visioncolor/Motion_estimation_and_Video%20coding_code.zip"
+
+imagenes:
+ - ruta: "coderde.webp"
+ titulo: "[Motion Estimation Scheme](#motion-estimation)"
+ descripcion: "Illustration of the video coder scheme with motion estimation and prediction error quantization."
+
+ - ruta: "coding.webp"
+ titulo: "[Video Coding Example](#video-coding)"
+ descripcion: "Comparison of video coding schemes with different levels of motion estimation and quantization efficiency."
+
+ - ruta: "distort.webp"
+ titulo: "[Distortion per Frame](#video-coding)"
+ descripcion: "Distortion per frame plot comparing algorithms with improved motion estimation and non-linear quantization schemes."
+---

content/code/image_video_processing/videoqualitytools/_index.md

@@ -2,10 +2,12 @@
 title: "VideoQualityTools"
 img: "FeatureImage_vreveal1.webp"
 image_alt: "VideoQualityTools Image"
-link: "../soft_visioncolor/subpages/video_quality.html"
+link: "./videoqualitytools/content"
 description: |
  VideoQualityTools is a Matlab Toolbox for perceptual video quality assessment based on the Standard Spatial Observer model augmented with Divisive Normalization. It performed second-best in VQEG Phase-I using no ad-hoc hand-crafted features.
 references:
  - "Importance of quantiser design compared to optimal multigrid motion estimation in video coding. Malo, J., Ferri, F.J., Gutierrez, J., and Epifanio, I. Electronics Letters, 36(9):807-809, 2000."
  - "Video quality measures based on the standard spatial observer. Watson, A.B., and Malo, J. ICIP, 2002."
+type: "code"
+layout: "single"
 ---