Tag: rsync

rsync logs with restricted ssh

Posted by – 15/04/2009

SSH is really the Swiss Army pocket knife of sysadmin tools. When I needed to periodically synchronize log files from an old server (old as in customer-would-never-update-it-or-install-anything-new), I built a simple and secure solution using rsync and ssh. This is what I did:

(I will call “remote” the system where the logs I want to retrieve are, and “local” system where I want them to be copied to) First I created an account with a restricted shell (ideally this should be a system account, but we’ll get there!):

remote# adduser --ingroup nogroup --shell /bin/rbash rlogs

Then locally, I created a new, password-less ssh key pair, copying it to my remote system:

local$ ssh-keygen
>>> When asked where to save it, I chose a different name, like .ssh/rlogs
local$ ssh-copy-id -i .ssh/rlogs.pub rlogs@remote
...
>>> You can delete the password of user rlogs, so it, effectively,
>>> cannot log-in with it (almost like a system user).
remote# passwd -d rlogs

Now you should be able to run password-less rsync already (note that I use -e option to point to a different key):

local$ mkdir logs
local$ rsync -av -e "ssh -i $HOME/.ssh/rlogs" rlogs@remote:"logs/" logs/
receiving file list ... done
./
file1
file2
...
fileN

But even with a restricted shell, I wanted even less possible things to happen. That’s what command= directive is for… It will only allow that command to be run in a session started by that key. Since rsync translates a lot of its command-line options, I run it again with a dirty ps-in-a-loop in the remote host, just to see what running rsync locally causes remotely:

remote$ while 1; do ps wp $(pgrep rsync); sleep 1; done
...
local$ rsync -av -e "ssh -i $HOME/.ssh/rlogs" rlogs@remote:"logs/" logs/
>>> in the remote loop you should be able to get the command:
  PID TTY      STAT   TIME COMMAND
 6183 ?        Ss     0:00 /usr/bin/rsync --server --sender -vlogDtpre.i . logs/

Here comes the authorized_keys magic. At the remote host I edited .ssh/authorized_keys to add a command= line with what I found out in my dirty loop. Also, I added a couple of directives to restrict it even further (they are pretty self-explanatory):

rlogs@remote$ cat .ssh/authorized_keys
command="rsync --server --sender -vlogDtpre.i . logs/",no-port-forwarding,no-agent-forwarding,no-X11-forwarding,no-pty ssh-rsa (...) myuser@local

Now everything is set. I just added the rsync command to the local crontab and it’s done.

Fork me on GitHub

Git basics: reversing the ‘git sucks’ effect

Posted by – 19/02/2009

I’ve been using git this last few days and I am still working on a workflow for my projects. Unfortunately, as others have noticed, git violates POLS is so many ways, it ends up being hard to get.

Creating a remote repository seems to be the first thing to bite a developer switching to git (mainly if coming from a centralized SCM). I have not decided which is the best way of doing it, but I’ve been using git-daemon via inetd and a path in my remote hosting holding all my repositories for public pulling, and ssh for pushing. Here is how to create it:

local$ ssh spectra@remotehost
remotehost$ mkdir /var/git/myproject.git
remotehost$ cd /var/git/myproject.git
remotehost$ git --bare init
remotehost$ logout
local$ cd myproject
local$ git remote add remotehost spectra@remotehost:/var/git/myproject.git
local$ git push remotehost master
...
otherlocal$ git clone git://remotelocal/myproject.git
otherlocal$ cd myproject
otherlocal$ git remote add remotehost spectra@remotehost:/var/git/myproject.git
(hack hack hack)
otherlocal$ git push remotehost master

Now everybody can pull your repository at git://remotehost/myproject.git and you can push and pull to it via ssh. Note that you have to setup git-daemon, which is pretty straight forward. I am using it as an inetd daemon, but you can use it as a standalone one. Debian has a package which does just that.

Now, some people think logging in a remote server just to create an empty repository is too much. Well… repositories are just .git directories. It happens that you can “push” for the first time by rsyncing your .git with a remote host:

local$ cd myproject
local$ rsync -a .git/ spectra@remotehost:/var/git/myproject.git
local$ git remote add remotehost spectra@remotehost:/var/git/myproject.git
(hack hack hack)
local$ git push remotehost master

(Apparently you can “push” using rsync every time, but it’s regarded as wiser to keep your – probably – crappy local repository commits separated from the public repository… otherwise commit messages like “Please, don’t use this code” are likely to pop up everywhere :) ). Now, I don’t know if this have side effects, but it works :-)

Another thing to notice is that git is more directed at pulling than pushing. This may be because of its designer: the way Linus works is by pulling changes from others’ repositories and not by letting others push’em into his one. And this is another violation of POLS for most of the people, who is used to “commit” their changes into some remote repository. Rather than that, people using git would expose their own repositories in order to have it pulled by others.

I also agree with most of the git critics wrt git commands… There’s a lot of examples – and I am not going deeper in this – but I think it was a bad choice calling “checkout” what git does when told to “checkout”, for instance. Yes, I know… different tools, different ways of seeing it… but everyone was already used to what centralized SCMs call their operations, and I think it would only help git adopting same names for the same operations, and inventing new ones for those proper of decentralized operations. Anyway, once you get it (and I have not completely got it yet), it seems all flow as expected. If this adaptation fails, there’s still Easy Git to the rescue!

One last thing that I think contribute to the “git sucks” effect: git-svn. This is a great tool, but it was built from git’s point of view… Given it’s intended as a glue for Subversion newcomers, it would benefit more from being built from svn’s point of view. This was mentioned by a colleague developer in my company, when he just couldn’t understand why one have to git svn rebase instead of a simple update. So git-svn also suffers of the “bad command naming habit” git do. Of course, that given that you came from this environment (I am sure git and git-svn makes perfect sense for Linus & cia :-) ). I have not tried yet, but yap seems to be targeted on providing an alternative to git-svn.