Hi all.
Was just thinking today about if it was ever considered to write the VTY history out to a file on exiting (and load it on startup)
I'm quite the fan of the up arrow and it's been something that sometimes annoys that I have no history after leaving the VTY.
It would seem reasonably trivial to implement, and I suppose you could write out to files in $HOME/.osmo-bsc_history or $HOME/.config or somesuch.
Just wondering if this was ever discussed before, if there's any unforseen problem with it on my part + if it would be a desirable feature for anybody else?
k/
On Sat, Oct 06, 2018 at 11:33:01AM +0100, Keith wrote:
Hi all.
Was just thinking today about if it was ever considered to write the VTY history out to a file on exiting (and load it on startup)
I'm quite the fan of the up arrow and it's been something that sometimes annoys that I have no history after leaving the VTY.
It would seem reasonably trivial to implement, and I suppose you could write out to files in $HOME/.osmo-bsc_history or $HOME/.config or somesuch.
Just wondering if this was ever discussed before, if there's any unforseen problem with it on my part + if it would be a desirable feature for anybody else?
k/
I like the idea. I have also missed this feature at times.
It should probably be opt-in because creating ever-growing files by default is a bit unfriendly to embedded platforms. Also, some commands accept crypto secrets in parameters, and people will not expect those to be saved by default under $HOME where any program could easily read them.
I don't see any other potential issues.
On Sat, Oct 06, 2018 at 12:52:07PM +0200, Stefan Sperling wrote:
It should probably be opt-in because creating ever-growing files by default is a bit unfriendly to embedded platforms.
you can just limit the size of those files and keep the last $N commands. Every shell manages quite fine with their history on that regard...
Also, some commands accept crypto secrets in parameters, and people will not expect those to be saved by default under $HOME where any program could easily read them.
As indicated $HOME is $HOME of the daemon, not the user. Anyone who has those privileges can just as well access the HLR database directly...
Hi Keith,
On Sat, Oct 06, 2018 at 11:33:01AM +0100, Keith wrote:
Was just thinking today about if it was ever considered to write the VTY history out to a file on exiting (and load it on startup)
Interesting idea!
I'm quite the fan of the up arrow and it's been something that sometimes annoys that I have no history after leaving the VTY.
It would seem reasonably trivial to implement, and I suppose you could write out to files in $HOME/.osmo-bsc_history or $HOME/.config or somesuch.
Not sure if $HOME is such a good idea for "system services/daemons" like the osmocom daemons. I wouldn't be surprised if that would either point to a read-only mount or /tmp in various configurations.
So it would probably rather be something in /var/lib/osmocom/$program-vty_history.txt or whatever...
$HOME/... would work better if we had an actual client program rather than just telnet. In that case, of course, it would be logical to store the history with the client, and not the server.
Just wondering if this was ever discussed before, if there's any unforseen problem with it on my part + if it would be a desirable feature for anybody else?
I don't think it has been discussed before.
The only problem that I can see is that there can be any number of users using any number of VTY sessions in parallel. So which history do we store? How do we keep them separate/disentangled? Or should we create one shared history?
Slighty unrelated: zebra/quagga also has many programs and each with their own VTY. They then introduced something called vtysh, whihc I only read about but never used myself. It supposedly allows you to talk to all of the various daemons from one frontend: https://www.systutorials.com/docs/linux/man/1-vtysh/ Definitely worth investigating if somebody is lookning for VTY usability improvements..
On Sat, Oct 06, 2018 at 12:54:04PM +0200, Harald Welte wrote:
On Sat, Oct 06, 2018 at 11:33:01AM +0100, Keith wrote:
Was just thinking today about if it was ever considered to write the VTY history out to a file on exiting (and load it on startup)
I've had the desire to implement this a number of times before. Especially because during testing, of course the vty session is closed with every program restart, and it takes extra effort to just call the same commands again...
Not sure if $HOME is such a good idea for "system services/daemons" like the osmocom daemons. I wouldn't be surprised if that would either point to a read-only mount or /tmp in various configurations.
[...]
The only problem that I can see is that there can be any number of users using any number of VTY sessions in parallel. So which history do we store?
I think it would have made more sense to do it per-user, but that isn't possible from the server process. We don't know which user is connected via telnet.
We could also toss up a telnet client that manages the history externally; osmo_interact_vty.py comes to mind, so far used mostly for transcript tests. With a few tweaks that could become a usable shell. Then we don't need to bother the server process with I/O of writing history files, and keep out the complexity of interleaving parallel VTY sessions.
Such a client could also have a bit of convenience for VTY port number lookups, like that tip of naming the ports in /etc/services, just without the need to edit that file.
Which sounds a lot like:
Slighty unrelated: zebra/quagga also has many programs and each with their own VTY. They then introduced something called vtysh, whihc I only read about but never used myself. It supposedly allows you to talk to all of the various daemons from one frontend: https://www.systutorials.com/docs/linux/man/1-vtysh/ Definitely worth investigating if somebody is lookning for VTY usability improvements..
~N
Hi all, thanks for the replies on this thread.
good to see some other people have considered it, and for pretty much the same reasons it occurred to me.
I tried rlwrap (the readline wrapper) with telnet, and of course it works, in that you get full readline support with saved history in your telnet session, on the client/user side) but you lose the [TAB] and [?] command completion of the vty.
I had a quick go and re-implementing the command completion using an rlwrap plugin, but my rusty perl skills did not allow me to complete that quickly, and I felt like whatever I might come up with would be klunky and hack-ish anyway, but It's probably possible. I think you have to stop rlwarp from sending a newline along with the command completion TAB though, and I'm not sure this is doable from the plugin. I'll have another look if I can find time.
I took a very quick look at zebra, (seems it's using sockets rather than tcp) and is making full use of readline. I guess a full "command and control centre" client application for the split osmo stack, might be an idea, but that's more effort than what I was proposing to do here, but I'd wonder if maybe a configurable "write history" option is not.
Of course, In my introspective viewpoint, I forgot that the server doesn't know which user is on the vty, and that there might be multiple users, I was only thinking of myslef, on my own local box. :/
But for such situations (which are quite common for devs?), would it make sense, to (as an option) just write the history on client disconnect to /var/lib/osmocom/[daemon-name].history or something.
I'll take a look at the python vty interact. I haven't looked at it.
Thanks!
k/
On Tue, Oct 09, 2018 at 01:46:53PM +0100, Keith wrote:
Hi all, thanks for the replies on this thread.
good to see some other people have considered it, and for pretty much the same reasons it occurred to me.
I tried rlwrap (the readline wrapper) with telnet, and of course it works, in that you get full readline support with saved history in your telnet session, on the client/user side) but you lose the [TAB] and [?] command completion of the vty.
All that needs to happen is that the tab and ? are sent to the telnet immediately, without waiting for the newline. Then it would reply with the completion / online doc output.
I had a quick go and re-implementing the command completion using an rlwrap plugin
you mean like navigate the vty tree and figure out what commands exist? that's not going to work, at least not like the real vty behaves. There is lots of magic going on there, argument range checking and whatnot.
I think you have to stop rlwarp from sending a newline along with the command completion TAB though, and I'm not sure this is doable from the plugin. I'll have another look if I can find time.
that sounds more like it. If we could only make all of [\n\t?] terminate a readline like \n does...
I'll take a look at the python vty interact. I haven't looked at it.
That would also need the don't-wait-for-newline stuff, and it currently also omits the prompts.
What works already is a kind of pseudo interactive shell: e.g. run osmo-bsc, then
$ osmo_interact_vty.py -p 4242 ?<enter> show Show running system information list Print command list exit Exit current mode and down to previous mode help Description of the interactive help system enable Turn on privileged mode command terminal Set terminal line parameters who Display who is on vty logging Configure logging no Negate a command or set its defaults
Needs to * show the prompt * change it to treat all of [\n\t?] as line-end; I hope that doesn't need re-invention of libreadline -- that's the biggest uncertainty. * write/read the history...
Hm, looks like it might end up a considerable effort after all.
~N
On 09/10/18 16:50, Neels Hofmeyr wrote:
On Tue, Oct 09, 2018 at 01:46:53PM +0100, Keith wrote:
I tried rlwrap (the readline wrapper) with telnet, and of course it works, in that you get full readline support with saved history in your telnet session, on the client/user side) but you lose the [TAB] and [?] command completion of the vty.
All that needs to happen is that the tab and ? are sent to the telnet immediately, without waiting for the newline. Then it would reply with the completion / online doc output.
The rlwrap plugin calls a function when TAB is pressed, completion_handler() and from there you can send what you will to the other side, and fill an array with completion words, that will be displayed and can be used for completion (and therefore sent with [CR])
https://github.com/hanslub42/rlwrap/wiki/RlwrapFilter.pm-manpage
I think it can be done, but you need to get the parsing of the vty output response to [TAB] right, and you need to avoid sending a [CR] along with the [TAB] or [?], which might require a patch to rlwrap.
On Tue, Oct 09, 2018 at 01:46:53PM +0100, Keith wrote:
Of course, In my introspective viewpoint, I forgot that the server doesn't know which user is on the vty, and that there might be multiple users, I was only thinking of myslef, on my own local box. :/
Multi-user history support could be left for later. For now, we could simply have the server write the history of any connected session to the file when command history is enabled. It is the simplest solution and works well for the most obvious use case, which is making repeated testing/debugging easier to do.
If multiple users are connected then their command histories will be combined. That's no different from what some shells are doing when a user opens multiple terminal windows at the same time.
A client-side history solution could still be added later, independently of any server-side solution.
On Wed, Oct 10, 2018 at 09:17:52AM +0200, Stefan Sperling wrote:
Multi-user history support could be left for later.
...is impossible to do server-side, without adding the concept of a user name, which I'd -1.
For now, we could simply have the server write the history of any connected session to the file when command history is enabled.
when a telnet vty closes, append all history to $history_path/$program_name.history ok, simple enough.
Yet it adds potentially large I/O to the server process, which would block until the history is written. That might be undesirable...
Could also flush the history to file every time it reaches a given size to make sure of light I/O load.
I sometimes invoke the vty with 'watch', or otherwise scripted, to keep showing the lchan summary, for example. That would produce a history entry for every single invocation, cause disk I/O and spam the history file. Do we need to enable history explicitly in each vty session? Or a a "no history" command per vty-session?
A client-side impl is much simpler from that angle. It just keeps so much cruft out of the server process, and "all we need" is a readline() that knows vty special characters.
A client-side history solution could still be added later, independently of any server-side solution.
I doubt we will have the other when one is already there.
So I guess it's up to who will contribute code first.
~N
On Wed, Oct 10, 2018 at 01:09:17PM +0200, Neels Hofmeyr wrote:
when a telnet vty closes, append all history to $history_path/$program_name.history ok, simple enough.
Yet it adds potentially large I/O to the server process, which would block until the history is written. That might be undesirable...
you cannot do a blocking write. The "history save procedure" would not be different and should be written via non-blocking file writes using the select() loop abstraction.
Could also flush the history to file every time it reaches a given size to make sure of light I/O load.
I don't expect the history of a human user at the VTY session every to reach a size where one would have to worry about that. We're talking about kilobytes here.
I sometimes invoke the vty with 'watch', or otherwise scripted, to keep showing the lchan summary, for example. That would produce a history entry for every single invocation, cause disk I/O and spam the history file. Do we need to enable history explicitly in each vty session? Or a a "no history" command per vty-session?
I would say explicit "no history"
A client-side impl is much simpler from that angle. It just keeps so much cruft out of the server process, and "all we need" is a readline() that knows vty special characters.
ACK. And we already have a [special] telnet client for nanoBTSs around on git.osmocom.org in http://git.osmocom.org/libtelnet/
I would currently argue more in favor of a client based solution
-
Harald Welte -
Keith -
Neels Hofmeyr -
Stefan Sperling