Documentation is needed

[awvwgk]

The documentation of the fpm-registry is currently not in a good shape. Good documentation is crucial to make a project like a package registry appealing for developers to actually submit their packages.

Currently, upon visiting the registry there are no guidelines on the scope and expectations of the registry, the packaging guidelines linked are for the package index at fortran-lang.org and not fpm specific. Especially the required keys for submitting to the registry are not documented.

Also, the package validation is tricky, the validation should happen with the load_registry.py script if I read the README correctly, but the actual validation of the fpm-enabled projects will only happen with build_index.py --check-new.

[milancurcic]

Thank you for starting this issue. I agree. I just looked at the README again and the first sentence says:

Centralized registry of fpm packages for the fortran-lang site.

I think this is misleading. fpm-registry is for much more than the website. The relevant thread on the scope is in fortran-lang/fpm#34.

the packaging guidelines linked are for the package index at fortran-lang.org and not fpm specific.

These are tests for scope and basic quality of a package. Do you have ideas about how to improve them of make them fpm-specific?

[awvwgk]

These are tests for scope and basic quality of a package. Do you have ideas about how to improve them of make them fpm-specific?

I agree, there is some overlap especially for the basic requirements. A first step would be a separate packaging guide in the fpm-registry repository, even if it basically contains the same information.

[awvwgk]

I tried to restructure the README and add a packaging guide but it feels like I'm getting nowhere with this effort. Frankly, I'm still not getting a good hold on this project and its purpose in the fpm ecosystem, which is odd, because the purpose should be right there in the name itself.

There were a few things that seemed confusing to me. The project is a collection of python scripts (load_registry.py, fpm_validate.py, fpm_fetch_toml.py, build_index.py), which seem to be standalones, yet they are importing each other (not very pythonic). Some of the scripts are for developer use, some are for automation, but this is not apparent from the naming. Also, the naming does not really reflect the purpose, the load_registry.py script actually validates if the registry.toml it is a valid TOML file (why?) and echos it in another format (purpose?). Trying to get the fpm_*.py scripts working to validate project outside of the build_index.py is also quite non-intuitive, yet they would be quite useful to check a project before submitting a PR.

To summarize, I'm struggling to connect the currently available infrastructure with the existing documentation and the apparent purpose of this project.

[milancurcic]

Yes, this project is a bit ahead of its time and the infrastructure is ad hoc. I will try to answer some of your questions.

I'm still not getting a good hold on this project and its purpose in the fpm ecosystem

The answer may vary depending on whom you ask, but IMO the purpose of fpm-registry is to enable a future (not yet implemented) capability of fpm to search for packages and add them to the manifest from the command line. For example, an fpm user may look for string packages, and do something like fpm search string, which would return a list of fpm packages (from this registry) that match the pattern. Once the user identifies a specific package that they'd like to use as a dependency, perhaps they would do something like fpm add toml-f, without having to know the git URL of toml-f, as that info would be pulled from the registry.

I think this repo is also used by the website (fortran-lang.org) scripts to mark specific Fortran packages as fpm packages there. This is a secondary purpose (IMO).

The project is a collection of python scripts

In my view, the key of this project is registry.toml. Python scripts were added mainly for CI, if I understand it correctly.

which seem to be standalones, yet they are importing each other (not very pythonic). Some of the scripts are for developer use, some are for automation, but this is not apparent from the naming. Also, the naming does not really reflect the purpose,

Yes, I agree, and short answer, this is all at a very early stage, implemented ad hoc to get something working quickly, and move on. It gets the job done. But I agree that it needs to be refactored and cleaned up before documenting it.

the load_registry.py script actually validates if the registry.toml it is a valid TOML file (why?)

If a user submits an incorrect entry to the TOML, this will catch it at PR.

and echos it in another format (purpose?).

Notice that registry.toml is a very minimal file, with no metadata other than project URLs. Python scripts fetch metadata from each of the packages and dump it into index.json. So when fpm search xyz that I described above is implemented, it will read the metadata from index.json so it can display package descriptions to the user.

I believe that the TOML was chosen for registry.toml, because it's easier to read than JSON, however I don't know why was JSON chosen for index.json. Perhaps it was due to generally wider support for parsing JSON than TOML from Fortran--recall that this repo started before we knew about toml-f.

Anyhow, that's my view. Thank you for wrestling with this. I think this would make a good GSoC project to refactor the tools, write extensive docs, and establish a workflow in this repo.

[ghost]

... For example, an fpm user may look for string packages, and do something like fpm search string, which would return a list of fpm packages (from this registry) that match the pattern. Once the user identifies a specific package that they'd like to use as a dependency, perhaps they would do something like fpm add toml-f, without having to know the git URL of toml-f, as that info would be pulled from the registry.

Hi @milancurcic . I'm working on this functionality (not yet published).

Current state is

• download index.json from fortran-registry: ✅
• parse index.json with json-c: ✅
• search package descriptions (case insensitive) with libpcre: ✅
• display the results to the user: ❌
• add package to fpm.toml: ❌

[awvwgk]

Thanks, for the clarification, having those information in the README would greatly improve the overall project in my opinion.

Regarding GSoC projects, I don't think this makes a good one until the documentation is improved and intent of this repo is made clearer, i.e. this issue is resolved.

[milancurcic]

@brocolis Great to hear, please keep us posted when it's ready to play with, I'm excited.

[sumit-158]

hey there, I'm new to open source but I would love to contribute in this issue so can I ?

[arjenmarkus]

@sumit-158, contributions, especially wrt documentation, are usually more than welcome ;), but this is a quiet week with people not being all that attentive. I guess that is why there has not been an answer before.

[sumit-158]

@sumit-158, contributions, especially wrt documentation, are usually more than welcome ;), but this is a quiet week with people not being all that attentive. I guess that is why there has not been an answer before.

@arjenmarkus I guess so, I have done some good first issue before but can you guide me how to get started with this issue I would be very thankful!!

[awvwgk]

This issue might be a bit more difficult to get started with, as this repository is a loose collection of scripts to maintain the current fpm registry. Describing the status quo of this repo will be a good start for the docs, but preferably we want to capture the actual intent.