Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring of the project to Typescript + Webpack #448

Open
wants to merge 54 commits into
base: master
Choose a base branch
from

Conversation

mattmoreira
Copy link
Contributor

This project has already served me a lot, and because of that, I wanted to somehow contribute to it, especially regarding its maintainability.

Given this, I've decided to develop an automated refactoring of the project, achieved by the usage of an AST parser called Recast, through which I could extract most of the root level classes and utility functions into different files.

I still have a problem with classes being declared and used in the same file, and usages of this before calling super, but other than that, it's almost done.

I've run the automated tests, and after those manual fixes cited above, it started working.

@jakubzitny, what do you think?

@jakubzitny
Copy link
Collaborator

jakubzitny commented Feb 3, 2020

Hi @mattmoreira, I read your e-mail, thanks for submitting this PR, I will try to have a look at it soon.

The problem with Evaporate's code "style" and architecture is really big and I agree that it would be awesome if we could split it properly, use the latest tooling and proper types and so on. From the quick glance at the changes, I think this auto-refactor will not be enough regarding the major changes. We could however, build on top of that and think properly about the future (where I'd like to finally resolve #437, use TypeScript, properly test everything and plug in CI), give me some time and I'll write up some proper feedback 😉

[Also we'd still need to get access to NPM releases and CI here to do proper job (shout out to @tomsaffell) who has been quiet for some time 😇 ]

@mattmoreira
Copy link
Contributor Author

@mattmoreira I'm using it right now in my project and it works very well!

Upload speed seems faster but the more impressive is the memory usage.
(Makes sense at least because now the variables are scoped.)

Maybe the memory leaks went away.

Very awesome! Maybe that's why I couldn't detect anything when trying to debug #437, as I was using this new TS version.

@mattmoreira
Copy link
Contributor Author

mattmoreira commented Apr 27, 2020

You might want to add Prettier to the project to normalize the code syntax.

Don't forget that you could take advantage of TypeWiz for automated types.

Added prettier, works over types using TypeWiz have started on https://github.com/mattmoreira/EvaporateJS/tree/chore/apply-types

@mtltechtemp
Copy link

mtltechtemp commented Apr 27, 2020

@mattmoreira Ok i'm on it too, lemme know if you need help on something.
Do you have Discord?

edit: TypeWiz has done something but the types names are horrible, we need to change that.

@mattmoreira
Copy link
Contributor Author

@mtltechtemp I think I was finally able to make it work! I deleted the old code and checked out the code from the master branch, and made the changes only on the bundle. I think it will work!

My Discord is MattElias#0591

@mtltechtemp
Copy link

@mattmoreira I'm still waiting for you to accept my request on discord.

@mattmoreira
Copy link
Contributor Author

mattmoreira commented Jul 26, 2020

Hey @jakubzitny, @TheWiseNoob and @mtltechtemp, I'm back!

Could you please review my PR for applying types to the project? pixelias-io#1

I've fixed the previous type names and applied a bunch of new types and interfaces as well.

* Initial codemod commit

* Fix global prefix and added importing of classes

* Add import global

* Renamed function for prefixing global identifiers

* Add exports to all files

* Fix constants not being imported

* Add example 3 for testing purposes

* Add more ignored classes

* Included lebab transformation on the project

* Ignored output of codemod

* Fix global prefixing while being in params

* Fix url being imported

* Change evaporate importing

* Add todo list of tasks to be done before finishing the transformer

* Add converter from js to ts

* Change require to import

* Refactored and split collection of class declaration and class usage

* Extract collectors to collector.js

* Refactor executeTransform, and fix transformClass to also add prototype expressions on nested classes

* Remove line causing issue on ts transformer

* Add clear script on prestart

* Add prettier and uncommented transform to ts

* Fix missing imports

* Remove ignore new expressions in favor of checking if the class was collected

* Commented es6code output

* Change lib compiler option to es7

* Add global blob interface

* Remove codemod output from gitignore

* Fix param overriding xhr instance

* Remove useless extend blob on types

* Add webpack bundling and analyzing

* Fix putPart throwing errors on file loader and constructor

* Change main on codemod to allow it to be imported by the test

* Fix evaporate module not being generated in the right format

* Add dist to gitignore

* Remove examples from codemod

* Change codeclimate to evaluate only the new code

* Add prepublish script to generate the bundle

* Add codemod to npmignore

* Change project version to 3.0.0-rc

* Fix webpack exporting umd with default and global variable with typo

* Change main file on package.json

* Add webpack for generating types

* Add npm scripts for collect and apply

* Started to apply typewiz

* Add prettier to the project

* Move value initializations to their class declaration

* Remove useless variables and constructor definitions

* Remove proprietary blob methods interface merging

* Fix spacing between util functions

* Move static methods to within the class

* Remove instrumented code

* Collect types with webpack-dev-server

* copy canonical files from master

* Change types contentBase from web to dist

* Fix slice on payloadFromBlob throwing exception

* Add mode development to make typewiz work

* Change apply types to work with web

* Applied the types collected and added some more

* refactored and add possible types to evaporate

* applied more types related to file upload

* add missing types to aws signature

* add more types to file upload

* add more types for each phase of the S3 part upload

* add type void to updateLoaded

* Created interface for callbacks

* add more types to file upload

* remove comment from file s3 part

* add types to putpart

* add types to signed s3 aws request

* add types to signed aws requests

* transform UploadHeadersInterface into compulsory and implemented it

* add aws signature types

* add util types

* add type to files in process

* add types to constructor in multipart upload

* add types on signed s3 request

* fix breaking polymorphism

* add types to history cache

* fix inference of awsRequest type

* fix canonical request returning empty

* fix not signed headers iteration

* fix the partNumberMarker type

* add more types to fileupload

* add type to authorize methdo on authorization custom

* replaced dictionary declarations

* replaced number by node timeout on putpart

* renamed types file

* change type on updateUploadFile to be partial

* add types to methods return in file upload

* made pause options more strict

* replaced properties typed with any on fileupload

* add missing finishedUploadingAt property on s3PartStats

* replaced more any values

* replaced more any types

* replaced history cache and log anys

* add missing aws key to file upload

* created logger type alias

* renamed and extracted some interfaces

* extracted the logger interface

* rename logger to logger interface

* extract evaporate status into a separate file

* Renamed createConfig to EvaporateConfigInterface

* rename EvaporateOverrideConfigOptions to EvaporateOverrideConfigInterface

* add more types to aws signature v4

* add types to aws signature v2

* add dictionary type on authorization method

* add type to history cache constructor

* add more specific dictionary type to headers

* add missing type to aws key on reuse s3 object

* add missing number type to aws signature v4

* removed typewiz
@mattmoreira
Copy link
Contributor Author

Thanks for reviewing @mtltechtemp! I'll do some final changes now and release another release candidate with the final code structure.

@mattmoreira
Copy link
Contributor Author

@jakubzitny, @TheWiseNoob and @mtltechtemp, I've added a PR to refactor the code structure, deleting the old evaporate file.

Can you please review it?

* moved code to src folder

* removed the rest of the codemod code

* exclude todo

* change main to dist

* bump package version

* add directories to package.json

* reorder package.json properties

* add back the webpack configs

* change timeout to number due to change from codemod

* add package lock

* rearranged directory structure on src

* changed webpack entry to match the new directory structure

* delete legacy evaporate js file

* moved put part to aws request folder

* moved multipart upload to aws requests

* move reuse s3 object to aws request
@mattmoreira
Copy link
Contributor Author

@jakubzitny, I've updated the upstream PR.

@mattmoreira mattmoreira changed the title Automated refactoring of the project, by using an AST transformer Refactoring of the project to Typescript + Webpack Aug 29, 2020
@mtltechtemp
Copy link

mtltechtemp commented Sep 14, 2020

@mattmoreira Works great for me!
It's been in production for about two weeks and no users are complaining about it.

Good work ! 👍

Edit: Safari, Firefox, Chrome are fine.

@pyr0hu
Copy link

pyr0hu commented Nov 8, 2020

Hey guys, I tried to install the latest rc

"evaporate": "github:TTLabs/EvaporateJS#pull/448/head", in package.json, but when I try to import it, I get the following webpack error: Module not found: Error: Can't resolve 'evaporate' in <file>.js. Any idea?

@camineroirving
Copy link

Hey guys, I tried to install the latest rc

"evaporate": "github:TTLabs/EvaporateJS#pull/448/head", in package.json, but when I try to import it, I get the following webpack error: Module not found: Error: Can't resolve 'evaporate' in <file>.js. Any idea?

I'm having the same issue in a typescript project, how were you able to fix the issue ? @pyr0hu

Thanks in advance.

@pyr0hu
Copy link

pyr0hu commented Apr 13, 2021

@camineroirving Sorry mate, could not resolve this so I just reverted to using the latest 2.1.4 release.

@TheWiseNoob
Copy link

Is this still being developed?

@stefanullinger
Copy link

Hi, I am also interested in the typescript version of this library. Having the same issue as pyr0hu and camineroirving though, when trying to use it in my project. Is there any plan to continue on this pull-request?

@AliMejbar
Copy link

is this still WIP ?

@Allan-Nava
Copy link

"Evaporate" is not exported by "node_modules/evaporate/evaporate.js" i have this problem

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants