Thoughts on a next generation MIB/VTY replacement

Hi All, Using zebra/quagga VTY code was a great idea back then and has served us nicely in all those years. I like the general style of the command-line based interface with its various nodes, the strict syntax checking, the tab completion and the interactive context-sensivive help. However, it feels a bit 20ieth-century-ish to have to manually write code to parse and to save the respective values. This is not a productive way to spend our development resources, and it is error prone. The "save" can be forgotten, resulting in non-saveable config parameters. The save can store values that the "parse" function will not be able to read again, etc. Furthermore, the VTY is inherently a human user interface, not intended for programmatic consumption. For programmatic access, we have developed the control interface. In reality though, only 1% of the parameters available in the VTY are exported via the control interface. And rather than adding all the missing bits and pieces with hand-crafted code to the control interface, we should have a generic way to define a new configuration value, and that value should then be automatically parse- and storable via VTY as well as a programmatic interface. The next "problem' is that the current telnet connections live within the process of the application. This means that a user can effectively "stall" the main application by performing I/O intensive operation on the VTY, or even on as many VTY/telnet sessions as he wants. There shouldn't be any need for this, at least not for something as mundane as performing configuration changes. Of course the VTY has different use cases such as runtime introspection of system state as well as logging. But still. Yet another concern is "VTY and telnet port proliferation". Particularly with the conversion from NITB to BSC+MSC+HLR, we are yet again getting more network elements with their own telnet ports. Remembering the port numbers is clumsy. It would be more convenient if a single command line interface could provide access to the configuration and the state of multiple different processes / network elements. Last, but not least, the current implementation is fixed to telnet, without any form of authentication, and without a path to migrate e.g. to something like SSL or SSH. I don't think it is a major concern (you can always SSH to the system and then telnet locally). So what I had in mind for quite some time (actually since netconf 1.2 about one year ago), is to have some kind of an external "VTY/MIB daemon" (or even separate daemons for each) which maintains a hierarchical database of configuration values. The MIB deamon simply offers an API (via client library) to GET or SET the individual values, or to NOTIFY an application about a changed value. This API is both used by the actual Osmocom programs to obtain their configuration (and obtain updates to it during runtime), as well as by the "VTY daemon" providing interactive shell access to it. Finally, other external applications could use the same interface/client library to do the same. A proxy to SNMP or REST interfaces can be imagined, for even more interfacing with the outside world. What I don't like about many existing MIBs I've used (as a sysadmin/user, not a developer) is their simple type system. You can often specify any random value to such a MIB, even one that is completely useless. A simple INT or STRING type is not sufficient, we really want the ability to have ENUM types, to have integers with ranges, etc. - just like we have in the VTY. Also, the MIBs I've used typically are nothing more than a hierarchical key-value store. They do not contain the information required for interactive, context-sensitive help needed for the "VTY" feel :( This is btw also the problem I have with OpenWRT's uci: It just has keys and values, with no way to constrain those values to something that's reasonable to the given program/context that is being configured, and there's no help or other information associated with those keys and values. So what I would like to see in this context, is some way to have a machine-parseable description of a "MIB/config item", together with syntax, ranges, enum values, help texts, etc (ideally in the C source code, possibly in comments?) which is used by some kind of MIB compiler to build up the hierarchical structure of the MIB and all of its keys as well as their permitted values and syntax reference. This information is then available to the VTY/telnet connections, irrespective of whether a given application [using those values] is running at all. Once an application starts, it can query for the configuration values it is interested in and populate its internal data structures from it. Ideally, one would even be able to generate a 'struct' from the MIB compiler, so that there is no need to explicitly call a "get" function on each single configuration value, but one simply gets a C-language 'struct' with all the values filled in by the client library. As stated above, there should be some way how he MIB can notify applications about changes to certain nodes in the MIB tree, so the application[s] can react to that. I don't have a clear picture yet how transaction logic (like changing multiple values and then committing them at once) would work, or whether applications should even be able to reject/revert a modification? In either case, I just wanted to share my thoughts on this. I know it sounds rather complex, but I think the investment in some kind of unified configuration database system would save us of a lot of boilerplate code and subtle bugs and inconsistencies. I've created https://osmocom.org/issues/1975 with above text to keep track of it, but I think at this point it's more of a mailing list discussion rather than something we can put into an issue. I guess the progress would rather be * first discuss it here and/or at OsmoDevCon * create something like a specification in the wiki * then follow-up with actual tickets on the work items All of this is brainstorming and vaporware, but I can't help to think of better ways of doing this. If somebody has experience with any existing systems and wants to share if and how they might fit, or what other ideas are useful to borrow: Please share! Regards, Harald -- - Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)