Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[sonic-cli-gen] first phase implementation of the SONiC CLI Auto-generation tool #1644

Merged
merged 83 commits into from
Nov 17, 2021
Merged

[sonic-cli-gen] first phase implementation of the SONiC CLI Auto-generation tool #1644

merged 83 commits into from
Nov 17, 2021

Conversation

vadymhlushko-mlnx
Copy link
Contributor

What I did

Implemented the sonic-cli-gen according to the SONiC CLI Auto-generation HLD

How I did it

How the sonic-cli-gen works:

  1. Parse provided YANG model to the intermediate representation - python dictionary (you can find the structure in the sonic_cli_gen/yang_parser.py file).
  2. Pass the python dictionary to the appropriate j2 template (config or show) and generate the CLI plugin.
  3. Put the auto-generated CLI plugin to an appropriate location, where it can be discovered by SONiC CLI.

How to verify it

Check the file with unit tests - tests/cli_autogen_yang_parser_test.py

Command output

admin@sonic: sonic-cli-gen generate config sonic-acl
Loaded below Yang Models
['sonic-acl', 'sonic-breakout_cfg', 'sonic-crm', 'sonic-device_metadata', 'sonic-device_neighbor', 'sonic-extension', 'sonic-flex_counter', 'sonic-interface', 'sonic-loopback-interface', 'sonic-port', 'sonic-portchannel', 'sonic-types', 'sonic-versions', 'sonic-vlan', 'sonic-vrf']
Note: Below table(s) have no YANG models:
COPP_GROUP, COPP_TRAP, FEATURE, KDUMP, MGMT_INTERFACE, SNMP, SNMP_COMMUNITY, WJH, WJH_CHANNEL, 
INFO:sonic-cli-gen: Auto-generation successful! Location: /usr/local/lib/python3.7/dist-packages/config/plugins/auto/sonic-acl_yang.py

admin@sonic: sonic-cli-gen remove config sonic-acl
/usr/local/lib/python3.7/dist-packages/config/plugins/auto/sonic-acl_yang.py was removed.

vadymhlushko-mlnx and others added 30 commits April 28, 2021 10:04
@vadymhlushko-mlnx
Worked version
548896a 
Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
@vadymhlushko-mlnx
added stub for function to determine static or dynamic YANG
3350f27 
Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
@vadymhlushko-mlnx
_find_index_of_yang_model()
1d1643a 
Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
@vadymhlushko-mlnx
added function to init list's for module, top, tables
cd31f0d 
Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
@vadymhlushko-mlnx
Added func to determine - static or list, +comments
81b803d 
Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
@vadymhlushko-mlnx
Added auto compleation file
ecdf8b6 
Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
@stepanblyschak
Add CLI Jinja templates
42779c7 
Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
@stepanblyschak
Fix the generated function name is incorrect
12bf0b2 
Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
@stepanblyschak
update and enhance templates with multiple lists in container
c8eaa66 
Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
@stepanblyschak
fix cli templates
dd29e33 
Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
@stepanblyschak
Merge branch 'master' of github.com:azure/sonic-utilities into sonic-…
0b0a0c1 
…cli-gen
@stepanblyschak
add clear list command
16857c4 
Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
@vadymhlushko-mlnx
Parsing for static YANG models - sonic-flex_counter.yang, sonic-devic…
dfa3ccc 
…e_metadata.yang DONE

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
@vadymhlushko-mlnx
Done refactoring for parsing 'static' YANG models
6b75f4b 
Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
@vadymhlushko-mlnx
Added parsing for 'dynamic' YANG models, done refactoring of whole code
d2be52b 
Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
@vadymhlushko-mlnx
Done parser for 'grouping' BUT need to deeply test it, added function…
73a3b12 
…s - get_leafs(), get_leaf_lists(), get_choices(), get_uses_grouping()

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
align with implementation
718f8ab 
Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
Merge branch 'sonic-cli-gen' of github.com:vadymhlushko-mlnx/sonic-ut…
647537c 
…ilities into sonic-cli-gen
generate cli from templates
fa6299c 
Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
@vadymhlushko-mlnx
Table can only have 'static-objects' OR 'dynamic-objects'
9d9f66b 
Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
add missing " in help for option
38f4365 
Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
Merge branch 'sonic-cli-gen' of github.com:vadymhlushko-mlnx/sonic-ut…
591aced 
…ilities into sonic-cli-gen
@vadymhlushko-mlnx
Added remove_keys(), get_mandatory()
0826ac5 
Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
fix issues
069000f 
Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
Merge branch 'sonic-cli-gen' of github.com:vadymhlushko-mlnx/sonic-ut…
594acce 
…ilities into sonic-cli-gen
@vadymhlushko-mlnx
Refactored generator.py: generate_cli_plugin(), get_cli_plugin_path()
1723f0f 
Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
@vadymhlushko-mlnx
Removed arguments from constructor of class CliGenerator, added funct…
0a78e19 
…ion comments

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
@vadymhlushko-mlnx
Added sceleton for UT, added additional check for yang_parser.py, cha…
30d2cb9 
…nged constructor for YangParser

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
@vadymhlushko-mlnx
Refactored _init_yang_module_and_containers()
0d371de 
Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
@vadymhlushko-mlnx
added 2 test cases
1593039 
Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
@stepanblyschak
Copy link
Contributor

@praveen-li Please check latest changes. Added type annotations in docstring.

praveen-li
praveen-li reviewed Oct 26, 2021
View reviewed changes
sonic_cli_gen/yang_parser.py Show resolved Hide resolved
sonic_cli_gen/yang_parser.py Show resolved Hide resolved
sonic_cli_gen/yang_parser.py Show resolved Hide resolved
sonic_cli_gen/yang_parser.py Show resolved Hide resolved
@stepanblyschak
use dict.pop() with default value
9e1854f 
Signed-off-by: Stepan Blyshchak <stepanb@nvidia.com>
@renukamanavalan
Copy link
Contributor

Code changes under sonic-utilities-data, looks good.

@stepanblyschak
if no data ion update_entry_validated() abort
f2677a1 
Signed-off-by: Stepan Blyshchak <stepanb@nvidia.com>
@stepanblyschak
if entry config did not change avoid calling set_entry()
4e0ac3e 
Signed-off-by: Stepan Blyshchak <stepanb@nvidia.com>
@stepanblyschak
pass a copy of cfg to FormatConverter.to_serialized() as it modifies …
6a7d7fa 
…original cfg

Signed-off-by: Stepan Blyshchak <stepanb@nvidia.com>
@vadymhlushko-mlnx
Copy link
Contributor Author

Hi @renukamanavalan and @praveen-li, did you finish the review?

@stepanblyschak
Copy link
Contributor

@renukamanavalan @praveen-li Could you please also review the second part of the feature in sonic-package-manager - #1650 ?

renukamanavalan
renukamanavalan reviewed Nov 11, 2021
View reviewed changes
sonic_cli_gen/generator.py Outdated Show resolved Hide resolved
sonic_cli_gen/generator.py Show resolved Hide resolved
@renukamanavalan
Copy link
Contributor

renukamanavalan commented Nov 11, 2021 •
edited
Loading

@renukamanavalan @praveen-li Could you please also review the second part of the feature in sonic-package-manager - #1650 ?

I am done with the review of this PR with few minor comments. I would like approval from Praveen, before we merge.
I am looking at 1650. @praveen-li, can you please help review too ?

@vivekrnv vivekrnv mentioned this pull request Nov 13, 2021
Merged
@liat-grozovik liat-grozovik mentioned this pull request Nov 14, 2021
Closed
praveen-li
praveen-li previously approved these changes Nov 16, 2021
View reviewed changes
Copy link

@praveen-li praveen-li left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now,

sonic-utilities-data/templates/sonic-cli-gen/config.py.j2 Show resolved Hide resolved
sonic_cli_gen/yang_parser.py Show resolved Hide resolved
sonic_cli_gen/yang_parser.py Show resolved Hide resolved
sonic_cli_gen/yang_parser.py Show resolved Hide resolved
sonic_cli_gen/yang_parser.py Show resolved Hide resolved
sonic-utilities-data/templates/sonic-cli-gen/config.py.j2 Show resolved Hide resolved
sonic-utilities-data/templates/sonic-cli-gen/config.py.j2 Show resolved Hide resolved
sonic_cli_gen/generator.py Show resolved Hide resolved
@vadymhlushko-mlnx
Added exception handling for j2 templates
afc4afb 
Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
@renukamanavalan renukamanavalan merged commit e7535ae into sonic-net:master Nov 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lguohan lguohan lguohan left review comments

@venkatmahalingam venkatmahalingam venkatmahalingam left review comments

@nazariig nazariig nazariig requested changes

@stepanblyschak stepanblyschak stepanblyschak approved these changes

@renukamanavalan renukamanavalan renukamanavalan approved these changes

@praveen-li praveen-li praveen-li left review comments

@ghooo ghooo Awaiting requested review from ghooo

Assignees
No one assigned
Labels
Included in 202111 Branch Request for 202111 Branch
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.