Discussion:
Y2038-safe replacements for utmp/wtmp and lastlog
Add Reply
Chris Hofstaedtler
2024-04-26 11:40:01 UTC
Reply
Permalink
Fellow Developers,

you are probably aware of the time_t-64bit migration :-)
However, this does not magically transition all data formats to 64bit
times. One such instance is the set of utmp/wtmp and lastlog files.

Thorsten Kukuk and others have been working on replacements for the
existing file formats and interfaces [1]; these are called wtmpdb
and lastlog2.

Some parties have requested that we do something in Debian [2]. If
we use Thorsten's work (and why not?), this likely means introducing
new packages into the Priority: standard set, and changes to a few
other packages, esp. those that handle user sessions.

Thorsten's code introduces new PAM modules to manage the new files,
so it should transparently work with most packages. Later, the
old interfaces can probably be turned off. This seems like a good
idea as a migration strategy to me.
A bonus seems to be that installs not wanting these features can
remove them - whereas today they are baked into everything.


On the wiki [0] I have summarized what I know; a list of initial
work items; and some open questions mostly concerned with upgrading.

I invite you to read the wiki page and the background info, to
identify gaps, to provide insights on feasability and further
related comments.
I'm hoping that we can build consensus on this plan.

Please keep #1068017 in CC: when discussing substantial matters
about this plan but drop it for only vaguely related sub-threads.

Chris


[0] https://wiki.debian.org/pam_lastlog2%20and%20wtmpdb
[1] https://www.thkukuk.de/blog/Y2038_glibc_lastlog_64bit/
[2] https://bugs.debian.org/1068017
Sam Hartman
2024-04-26 13:30:01 UTC
Reply
Permalink
Chris> Fellow Developers,
Chris> you are probably aware of the time_t-64bit migration :-)
Chris> However, this does not magically transition all data formats to 64bit
Chris> times. One such instance is the set of utmp/wtmp and lastlog files.

Chris> Thorsten Kukuk and others have been working on replacements for the
Chris> existing file formats and interfaces [1]; these are called wtmpdb
Chris> and lastlog2.

Relatedly, PAM upstream and apparently at least some aspects of Fedora
have been moving to logind to handle utmp functionality.
You will start to see the first impacts of that in pam unstable.

--Sam
Luca Boccassi
2024-04-26 15:00:02 UTC
Reply
Permalink
Post by Chris Hofstaedtler
Fellow Developers,
you are probably aware of the time_t-64bit migration :-)
However, this does not magically transition all data formats to 64bit
times. One such instance is the set of utmp/wtmp and lastlog files.
Thorsten Kukuk and others have been working on replacements for the
existing file formats and interfaces [1]; these are called wtmpdb
and lastlog2.
Some parties have requested that we do something in Debian [2]. If
we use Thorsten's work (and why not?), this likely means introducing
new packages into the Priority: standard set, and changes to a few
other packages, esp. those that handle user sessions.
Thorsten's code introduces new PAM modules to manage the new files,
so it should transparently work with most packages. Later, the
old interfaces can probably be turned off. This seems like a good
idea as a migration strategy to me.
A bonus seems to be that installs not wanting these features can
remove them - whereas today they are baked into everything.
On the wiki [0] I have summarized what I know; a list of initial
work items; and some open questions mostly concerned with upgrading.
I invite you to read the wiki page and the background info, to
identify gaps, to provide insights on feasability and further
related comments.
I'm hoping that we can build consensus on this plan.
Please keep #1068017 in CC: when discussing substantial matters
about this plan but drop it for only vaguely related sub-threads.
Chris
[0] https://wiki.debian.org/pam_lastlog2%20and%20wtmpdb
[1] https://www.thkukuk.de/blog/Y2038_glibc_lastlog_64bit/
[2] https://bugs.debian.org/1068017
Would be nice to drop things that are not used, but otherwise, option
A looks good and broadly similar to what other distros are doing, so
should be pretty safe. Thanks for taking care of this.
RL
2024-04-26 19:10:01 UTC
Reply
Permalink
Post by Chris Hofstaedtler
you are probably aware of the time_t-64bit migration :-)
However, this does not magically transition all data formats to 64bit
times. One such instance is the set of utmp/wtmp and lastlog files.
Thorsten Kukuk and others have been working on replacements for the
existing file formats and interfaces [1]; these are called wtmpdb
and lastlog2.
Some parties have requested that we do something in Debian [2]. If
we use Thorsten's work (and why not?)
Thorsten's code introduces new PAM modules to manage the new files,
so it should transparently work with most packages. Later, the
old interfaces can probably be turned off.
On the wiki [0] I have summarized what I know; a list of initial
work items; and some open questions mostly concerned with upgrading.
I invite you to read the wiki page and the background info, to
identify gaps
the chkrootkit package provides several utilities for examining some of
these files: chkutmp chkwtmp and check_wtmpx and chklastlog [a] -- it does
not use pam but reads the files in /var/log

How would I test these against the new files - i assume the new versions
are compatable but might need bigger variables in those utilities? (any
assistance with that review is welcome - C is not my thing!)

[a] You can read these here ---
https://salsa.debian.org/pkg-security-team/chkrootkit but nb that there
are many patches in debian/patches that touch these (use gbp pq import
to see the patched versions)
Post by Chris Hofstaedtler
[0] https://wiki.debian.org/pam_lastlog2%20and%20wtmpdb
[1] https://www.thkukuk.de/blog/Y2038_glibc_lastlog_64bit/
[2] https://bugs.debian.org/1068017
Richard
Jun MO
2024-05-07 23:10:01 UTC
Reply
Permalink
Dear Developers,

A bit from a Debian user. Please note that I am not come to here to
blame/complain
against the Upstream/Maintainer of the pam package and the Maintainer of
the shadow
package, or come to here to request something. I just come to here to
present
some of my hope.

I often use the w(1)/last(1) command, and sometimes use the lastb(1)
command.
Several days ago, I noticed that records logged in from
console(tty{1..6}) are missing
from the output of last(1) while records from ssh/tmux/lightdm are still
there,
and I started to guess maybe some of my recent changes to the system
caused this.
Today I try to find out what happened, and after several hours of
fruitless effort
(try different options of agetty/login, use strace/gdb on agetty/login
and read
the source of the shadow package), I noticed that a word "pam_lastlog"
in the
source code. Finally I find out the problem is caused by that login(1) use
pam_lastlog.so to write /var/log/wtmp, but pam_lastlog.so was not longer
included
in the libpam-modules package. (It was somehow related to #1068229, but
I missed
it.)

From my understanding, why we need to deal with the Y2038 issue is that
the issue
may cause problems, some of which will be big problems, after year 2038.
But so,
let's now rush some changes, to confuse users or break something? (Just
joke and,
again I am not to blame anyone.) Are there issues using
utmp/wtmp/lastlog and
w(1)/last(1)/lastlog(1), *currently*? Are there security issues, big
defects or
are they hard to maintain?

If not, I prefer a bit more slow pace, compatible and less disturbed
process.
More specifically, regarding to some changes proposed in the wiki [1],

1) I hope there will still be the original
w(1)/last(1)/lastb(1)/lastlog(1)/faillog(1)
tools which can still read *old* format utmp/wtmp/lastlog in Debian at
least for
a while after switch to Y2038-safe replacements. Those tools can read
old format
files without convert/import to new format. I am keeping old wtmp files for
several years. Starting from 2016 my system with a proprietary nvidia
driver failed to
resume from suspend2ram and playing a video using hardware accelerator would
cause the system unstable. Five years later, still having the problem,
with some
help of reading kernel version from `last -f /var/log/wtmp.*', I finally
found
out that there is a commit change in the kernel caused the problem. This
shows
that having those tools installed may provide a few help. Another point
may be
that package from 3rd parties may still write old-format
utmp/wtmp/lastlog, it
will be good still having those tools installed at least for a while.
Those tools
may be modified that when invoked, print messages to inform users
time/date maybe
incorrect after the year 2038, and suggest/recommend/urge users only use
those
tools to read old files and switch to use new replacements. Anyway it seems
keeping those tools have little harm. And I have a look into wtmpdb,
from salsa[2].
From manpage and --help, it seems to me the current version 0.11.0 of
wtmpdb does
not support read/import/migrate from /var/log/wtmp, the suggest of
/usr/bin/wtmpdb
take over last(1) in the wiki is not feasible as some user may still
expect using
`last -f /var/log/wtmp.*' to read old files. Even if new version of
wtmpdb can
read from /var/log/wtmp without import it into
/var/lib/wtmpdb/wtmpdb.db, it is
still good to have a choose to use the original last(1) to read from
/var/log/wtmp.
(Also see below.) It is similar to lastlog2. I see lastlog2 can already
import
from /var/log/lastlog, but from usage() [3], it will always import to a
lastlog2
database.
2) I hope I can choose keeping or deleting the old utmp/wtmp/lastlog
files when
switch to Y2038-safe replacements. By default, those old files may be
automatically
deleted, but before extracting new package/running maint scripts, there
may be a prompt
telling users that those files will be deleted and asking users chose
whether continue
or not; if not, dpkg should exit without deleted those files. Or those
will not be
automatically deleted, instead a Debian.NEWS may be displayed or maint
script can
print a message telling those files can be safely deleted after
converted. I think
the current dpkg already have functions to implement aforementioned
actions as I
have seen something alike many times. I known normally purge a package
will deleted
its log(and configuration), but wtmp/btmp/lastlog/faillog do not belong
to any
package and many program can read from/write to them. Also it seems to
me that
deleting logs during upgrade is not so good, and maybe leave it to users
to decide.
You may ask why I want to keep those old format files instead of
converting them
and use new tools to read? I can not exactly tell why, but I maybe
afraid the
converting may not perfect and I want to compare both output before
deleting the
old ones. For example, the old format files may have corrupted, and the
converting
silently skip what it can not parse. Or for no reason, I just want keep
those
file in a original intact form.

So for summary, I hope original last(1)/lastlog(1) etc. can still in
Debian after adopted
Y2038-safe replacements and I could chose whether to delete
utmp/wtmp/lastlog files
or not before upgrade a package. Or more generally, I hope changes can
be performed
in a compatible and less disturbed process, or if it can not be avoided,
at least
I could be informed and choose what to do before it happened. It will be
very
appreciate if these can be take into consideration before decides being
made.

Another note, in my system, I found some packages do not use PAM to
write utmp/wtmp.
screen/tmux/xfce4-terminal depend on libutempter0 which provide a
library and
a setgid helper to write utmp/wtmp. The runit package provides
utmpset(8) which
can wirte a logout record to utmp/wtmp. Once in Debian but removed a
while ago,
the sac package has writetmp(1)/rawtmp(1) which can also write to wtmp.
These package, maybe among others, may be needed to modify to write to
Y2038-safe replacements.

Last, thanks for your works and your reading.

Regards,
Jun MO

[1] https://wiki.debian.org/pam_lastlog2%20and%20wtmpdb
[2] https://salsa.debian.org/debian/wtmpdb
[3]
https://github.com/util-linux/util-linux/blob/926b6077333554924756ba648c9df38c803c48bc/misc-utils/lastlog2.c#L103
Craig Small
2024-05-07 23:30:01 UTC
Reply
Permalink
Post by Jun MO
1) I hope there will still be the original
w(1)/last(1)/lastb(1)/lastlog(1)/faillog(1)
tools which can still read *old* format utmp/wtmp/lastlog in Debian at
least for
a while after switch to Y2038-safe replacements. Those tools can read
I can only speak for w. It currently prefers what it gets from systemd or
elogind, effectively
iterating over the sessions coming from sd_get_sessions() if sd_booted()
returns true.

If sd_booted() returns false, then it uses the old utmp/utmpx files for
now. Besides the Y2038
issue, the utmp "API" is pretty awful with things like errors pretty much
undetectable. There is also
the problem about who (e.g. which process) should be writing to those files
as you have pointed out
in your email.

For now w/uptime will use utmp as a fallback, but I'll be happy if this
gets updated to something better; it's a low-priority
for me because systemd/elogind do what I need most of the time.

- Craig
Jun MO
2024-05-08 09:20:01 UTC
Reply
Permalink
I can only speak for w.  It currently prefers what it gets from
systemd or
elogind, effectively
iterating over the sessions coming from sd_get_sessions() if sd_booted()
returns true.
If sd_booted() returns false, then it uses the old utmp/utmpx files for
now. Besides the Y2038
issue, the utmp "API" is pretty awful with things like errors pretty much
undetectable. There is also
the problem about who (e.g. which process) should be writing to those files
as you have pointed out
in your email.
For now w/uptime will use utmp as a fallback, but I'll be happy if this
gets updated to something better; it's a low-priority
for me because systemd/elogind do what I need most of the time.
Thanks for explaining.

For last(1) my concern is that it will be helped to keep the original
last(1) for back-compatibility to read old wtmp files. For w(1), utmp
is only about current sessions, so there is no need to keep years-old
utmp files. Unlike last(1), there is no something like `w -f /run/utmp'.
Actually, one can run `last -f /run/utmp', and it provides output
similar to w(1)'s except missing something like process and CPU times
for each user. And as pointed out by you, w(1) currently already prefer
using infos from systemd/elogind instead of reading from utmp.

So I now think w(1) may be little need to keep the ability to read from
utmp and am also happy to it can change to use something better.

Regards,
Jun MO

Loading...