Skip to content

Latest commit

 

History

History

task3

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
ChernikovTask3.ipynb
ChernikovTask3.ipynb
 
 
PetrushinaTask3.ipynb
PetrushinaTask3.ipynb
 
 
Protasov_Dmitry_ENAS.ipynb
Protasov_Dmitry_ENAS.ipynb
 
 
bmm_genetic_boeva.ipynb
bmm_genetic_boeva.ipynb
 
 
readme.md
readme.md
 
 

readme.md

Deadline: March 1, 23.59.

Save notebooks into task3/SurnameTask3.ipynb

IMPORTANT: the code must not be written in Torch/Tensorflow. For deep learning use Jax.

  1. [reporter: Timofey Chernikov] Implement 1-st order DARTS for CIFAR-10 dataset. Compare with random structure selection. Model: 3-layer network. Structure: binary mask for each parameter. Plots: model peformance, mask. NOTE: binary mask won't work with unconstrained parameter optimization, think how to overcome this limitation.

  2. [reporter: Dmitry Protasov] Implement ENAS for CIFAR-10 dataset. Compare with random structure selection. Model: 3-layer network. Structure: binary mask for each parameter. Plots: model peformance, mask.

  3. [reporter: TODO] Implement naive-RMAD (without information loss restoring) for 2d problem (the loss surface must be nontrivial). Plot the forward trajectory and restored trajectory for different number of epoch and different momentum (mu = 0, 0.5, 0.9, 0.99).

  4. [reporter: Parviz Karimov] Consider Bayesian linear regression. Restore covariance matrix using Bayesian optimization methods. The covariance matrix must be full-ranked. Plot the dependence of the restoration performance from number of iterations for different data dimmensions.

  5. [reporter: Marat Khusainov] Consider Bayesian linear regression. Restore covariance matrix using HOAG method. Consider full-ranked and 1-ranked covariance matrix. Plot the dependence of the restoration performance from number of iterations for different data dimmensions.

  6. [reporter: TODO] Implement greedy hyperparameter optimization for logistic regression on FASHION-MNIST. Compare two types of hyperparameters:

    • Binary mask for each parameter
    • l2-coefficients for each parameter.

    NOTE: binary mask won't work with unconstrained parameter optimization, think how to overcome this limitation.

    Compare results and hyperparameters.

  7. [reporter: Boeva Galina] Implement a genetic algorithm for model structure selection for CIFAR-10 dataset. Compare with random structure selection. Model: 3-layer network. Structure: binary mask for each parameter. Plots: model peformance, mask.

  8. [reporter: TODO] Implement a ENAS-based symbolic regression.

  9. [reporter: Kseniia Petrushina] Consider Bayesian linear regression. Restore covariance matrix using Bayesian optimization methods. The covariance rank matrix must vary from 1 to full-rank. Plot the dependence of the restoration performance from number of iterations for different ranks.