Typescript integration for Meteor packages

[piotrpospiech]

Hi, as a part of the Vazco & Meteor Software cooperation, I've just started working on native TypeScript type definitions support for Atmosphere packages and I want to share with you some details.

Currently, packages installed with Atmosphere are missing type definitions. Even if a source code would have them, it won't be used in your IDE. For this reason, we use DefinitelyTyped to support Atmosphere packages with type definitions. My goal is to remove the necessity to use an external library and import all types, that will be provided with the source code, to the project. That will help to keep TypeScript types up-to-date and improve a developer's experience.

The roadmap looks as follows:

1. Proof of concept
   • Atmosphere packages are located in the ~/.meteor/packages. I will create an external script that will copy index.d.ts from the package source code to the project
   • It needs to define what package version is used on the project and it should localize the directory with its source code
   • At the beginning, it will be working with one specific package - universe:i18n
   • Proper TypeScript configuration will be needed, so I will provide documentation of what changes will be required in the tsconfig.json
2. Release candidate
   • Script should be integrated with Meteor to do it automatically. I still need to do some research about it, but there are the main ideas for now:
     • It can be done after starting the project. However, it probably has more disadvantages than advantages because it's time-consuming and not always needed.
     • Also it can be run after updating Atmosphere packages i.e., installing, updating, and removing.
   • Currently, TypeScript type definitions are stored in the DefinitelyTyped repository. A requirement for this to work natively in Meteor is that Atmosphere packages should have index.d.ts files in their main repository
3. Stable release
   • Script should find more than one file with type definitions. I will provide support for all .d.ts files in the package repository
   • More packages will be integrated with this feature

Feel free to comment and share your opinion.

References:

• Atmosphere packages with typescript definitions?
• How to declare "meteor.d.ts or index.d.ts" in meteor package? #10828
• [1.3] - How to prevent TypeScript from marking the meteor/PACKAGE_NAME imports as error? #6177
• Support for building and distributing Typescript types for Meteor Packages #11735

[copleykj]

So I was actually giving this a bit of thought the other day myself and had very similar thoughts. Here was what I was thinking.

1. Meteor's build tool should package all .d.ts files and include them in the iso build bundle under a types directory
2. Create a CLI interface that can be called by things like npm lifecycle scripts such as postinstall
3. CLI should analyze ./meteor/versions finding the currently used packages and versions
4. CLI should use Meteor's package cache, which now thanks to the build took contains a new types directory for each package version, and copy each package's types at the currently installed version (e.g. ~/.meteor/packages/univers_i18n/1.36.2/types)
5. CLI should place each type in a corresponding folder in `./node_modules/meteor' (e.g node_modules/meteor/universe:i18n/)
6. CLI should be called after meteor add, meteor remove, and meteor update (could we create some hooks in the meteor tool for this?) as well as by the 'postinstall` npm script.

There are some some caveats here like the fact that npm sometimes overwrites the node_modules dir in its entirety. This is mitigated by using the CLI to re-copy the types using the postinstall script, and other projects such as Prisma are using this approach to have a dynamically generated api that lives in the node_modules dir.

Hopefully this might give ideas that you might not have had yet. I'm very excited for this as this opens a whole realm of possibility for Meteor.

One other thing we might do is document a best practice for setting up the a package to generate the type defs before publishing. Maybe this could even be a "beforePublish" hook for the meteor tool that specifies a command to run before the publishing

  "meteor": {
    "mainModule": {
      "client": "client/main.tsx",
      "server": "server/main.ts"
    },
    "testModule": "tests/main.ts",
+   "hooks": {
+    "beforePublish": "tsc index.js --declaration --emitDeclarationOnly",
+    "afterPackageModification": "meteor-ts",
+   }
  },

[fredmaiaarantes]

I reached out to @zodern to figure out how we can collaborate on this feature.

Nice! Pls keep us posted. Ideally, we should bring this feature into our TypeScript package and not require extra steps for developers.

[zodern]

I agree the integration should eventually be part of Meteor. I thought it would be good to initially experiment as a non-core package so we can try more solutions and better understand any problems first. Also, I have a few packages I want to add types to that support old Meteor versions that don't have the typescript package, so they would likely need a non-core solution anyway for publishing them.

Regarding zodern:types, it seems to mostly work. The known issues are:

• The typescript language server used by vscode and some other code editors doesn't always detect when the types for a package are updated. Restarting the language server or code editor fixes it.
• Linter build plugins are slower than necessary and have some bugs. There are work arounds the package can use, and I also plan to submit a PR to Meteor to fix them
• For some local packages it uses a .ts instead of .d.ts file. It seems typescript has the package's tsconfig.json inherit parts of the app's tsconfig.json, which might make it more strict and create type errors in the package's code.

Things it is missing:

• types when importing a specific file from a package (for example: import { addStory } from 'meteor/zodern:aurorae/react')
• setting up types for a package's dependencies on other Meteor packages so when running tsc for the package it can find the types
• I haven't looked into how typing the globals a package exports would work

I think the next steps are:

• Use zodern:types in apps and packages to see what issues come up
• Possibly explore other options of having typescript find a package's types. The code for the current implementation is in writer.js
• Decide how to pick which file to use for a package's types. The code for this is in types-entry.js.
  • Decide where types should be within a package. When packages have zodern:types generate types, it currently does this.
• Figure out how to provide types for when an app imports a specific file from a package
• See what happens when Meteor packages use types from npm packages, and figure out how to better handle this and fix any issues with it.
• If we decide to stick with having a single packages.d.ts file, look into creating a source map for it so code editor's Go to definition feature jumps directly to the package's types instead of to the packages.d.ts file.

[filipenevola]

@zodern this is great.

I believe it would be better to have it as part of the core asap so core packages could also switch to this way of declaring types.

I don't see any issues, except for really old packages without typescript support, of having it in the core since day 0.

---

About VSCode, maybe you should talk with @matheusccastroo to see if Meteor Toolbox Extension could help somehow.

[zodern]

We actually can start adding types to core packages now. You can do it by adding a single .d.ts file to the package, and add typescript as a dependency so Meteor includes the .d.ts file when publishing. A couple things we need to figure out for core packages to have types are:

1. How to have typescript prefer the types from core packages over those from @meteor/types. I had this working initially in zodern:types, but the approach it used caused other problems so I removed it.
2. How to type globals the package exports

We likely will run into other problems, so this would be a good way to check if the solution zodern:types uses would work, or if we need to find a different one. There is still a lot I don't know about typescript, but I am wondering if the current solution could only cover 80% - 90% of what we need. For example, I'm not sure it will be possible to add types for module.hot.

[radekmie]

I think I can help here:

How to have typescript prefer the types from core packages over those from @meteor/types. I had this working initially in zodern:types, but the approach it used caused other problems so I removed it.

I simply wouldn't. We can move all @types/meteor to their corresponding packages and tell people to uninstall @types/meteor. Then, as a next step, integrate them in the code instead of having a single .d.ts file next to the source.

How to type globals the package exports

That's solved in @types/meteor too. There's a script, that takes all of the "standard modules" (i.e., the ones using declare module 'meteor/X') and transforms them into "globals" (i.e., uses declare namespace). And yes, you have to have both in order to make it work. (Technically, we could try to make the global ones use modules, but that's kind of tricky; I already tried it for @types/meteor twice.)

For example, I'm not sure it will be possible to add types for module.hot.

Why? @types/webpack-env does that.

[radekmie]

Starting v2.8.1 Meteor now offers its own types thanks to zodern:types, see migration steps for more information.