Consolidating wtmp log across all login nodes on Rocks clusters: run a command only for the first login in a given day across all the login nodes

Our stakeholders wanted to see their quotas and usage on different filesystems as soon as they logged in onto cluster. We have 4 login nodes and users get distributed onto them in round robin fashion (round robin DNS).

So, I wrote a script in bash and added few lines of code to /etc/profile and /etc/csh.login to run this script whenever users login. Everything was great except that this script was running for everyone and every time they logged in. Not ideal.

Then, I wanted to make sure this script would not run for user root and also when root becomes another user via su. Then, I wanted to show the quota only once a day. So, I changed the code again on /etc/profile and /etc/csh.login. Finally, users would see it once a day on any given login node. Still, I wasn't happy because if user logged in multiple times, s/he would get onto different login nodes. This means, s/he would see the quota & usage information at least 4 times if s/he logged 4 times and got onto 4 different login nodes.

I wanted to consolidate wtmp logs from all login nodes and use that information. After making subtle changes, finally everything seems to work well. Since I am consolidating wtmp every minute via a cron job, the only time this script doesn't work is when a user logs in multiple times with in a minute and s/he ends up on different login nodes. In this case, s/he would still see the quota usage information.

On all login nodes we need to add this code at the end of /etc/profile to make myquota run only for the first login in a day for any user.

if [ $USER != root -a `id -G $USER | cut -f1 -d" "` -ne 10 -a $(who am i | awk '{print $1}') = $USER ];then
 if [ -z "$(last -2 -f /share/apps/admins/wtmp.merged $USER | awk 'NR==2')" ];then
  myquota
  echo -e "See this report at any time with `tput bold`'myquota'`tput sgr0`\n"
 else
  if [ $(last -F -f /share/apps/admins/wtmp.merged $USER | sort -k1.48,1.49n | awk 'END{print $6}') -ne $(date '+%_d') ];then
   if [ -z "$(last -2 $USER | awk 'NR==2')" ];then
    myquota
    echo -e "See this report at any time with `tput bold`'myquota'`tput sgr0`\n"
   else
    if [ $(last -2 $USER | awk 'NR==2{print $6}') -ne $(date '+%_d') ];then
     myquota
     echo -e "See this report at any time with `tput bold`'myquota'`tput sgr0`\n"
    fi
   fi
  fi
 fi
fi

On all login nodes we need to add this code to end of /etc/csh.login to make myquota run only for the first login in a day for any user.

if ( $USER != root && `id -G $USER | cut -f1 -d" "` != 10 && `who am i | awk '{print $1}'` == $USER ) then
 if ( "`last -2 -f /share/apps/admins/wtmp.merged $USER | awk 'NR==2'`" == "" ) then
  myquota
  echo "See this report at any time with `tput bold`'myquota'`tput sgr0`"
  echo
 else
  if ( `last -F -f /share/apps/admins/wtmp.merged $USER | sort -k1.48,1.49n | awk 'END{print $6}'` != `date '+%_d'` ) then
   if ( "`last -2 $USER | awk 'NR==2'`" == "" ) then
    myquota
    echo "See this report at any time with `tput bold`'myquota'`tput sgr0`"
    echo
   else
    if ( `last -2 $USER | awk 'NR==2{print $6}'` != `date '+%_d'` ) then
     myquota
     echo "See this report at any time with `tput bold`'myquota'`tput sgr0`"
     echo
    endif
   endif
  endif
 endif
endif

On the master node, I have this in my crontab.

* * * * * (/opt/rocks/bin/tentakel -glogin 'cp /var/log/wtmp /share/apps/admins/wtmp.$(hostname -s)';/bin/cat /share/apps/admins/wtmp.login-0-[0-3] > /share/apps/admins/wtmp.merged)>/dev/null 2>&1