root is god
selinux is blasphemy
Sorry for the long silence. I will try to post more often from now on.
I playing around with dnf, which is the package manager that should replace yum in the future and had some very nice insight.
[root@mysystem ~]# date ; yum clean all Do 25. Sep 17:51:06 CEST 2014 Geladene Plugins: auto-update-debuginfo, fastestmirror, langpacks, ps, refresh- : packagekit, remove-with-leaves, rpm-warm-cache, show-leaves, : versionlock Quellen werden aufgeräumt:adobe-linux-x86_64 fedora fedora-HandBrake : fedora-chromium-stable logstash-1.4 : mbooth-eclipse-luna rpmfusion-free : rpmfusion-free-updates rpmfusion-nonfree : rpmfusion-nonfree-updates updates Cleaning up everything [root@mysystem ~]# date ; yum update Do 25. Sep 17:51:13 CEST 2014 Geladene Plugins: auto-update-debuginfo, fastestmirror, langpacks, ps, refresh- : packagekit, remove-with-leaves, rpm-warm-cache, show-leaves, : versionlock adobe-linux-x86_64 | 951 B 00:00 fedora/20/x86_64/metalink | 33 kB 00:00 fedora | 3.8 kB 00:00 fedora-HandBrake | 2.9 kB 00:00 fedora-chromium-stable | 3.4 kB 00:00 logstash-1.4 | 2.5 kB 00:00 mbooth-eclipse-luna | 3.0 kB 00:00 rpmfusion-free | 3.3 kB 00:00 rpmfusion-free-updates | 3.3 kB 00:00 rpmfusion-nonfree | 3.3 kB 00:00 rpmfusion-nonfree-updates | 3.3 kB 00:00 updates/20/x86_64/metalink | 15 kB 00:00 updates | 4.9 kB 00:00 (1/8): fedora-HandBrake/20/x86_64/primary_db | 10 kB 00:00 (2/8): fedora/20/x86_64/group_gz | 394 kB 00:00 (3/8): mbooth-eclipse-luna/20/x86_64/primary_db | 148 kB 00:00 (4/8): updates/20/x86_64/group_gz | 395 kB 00:00 updates/20/x86_64/primary_db FAILED http://mirror.1000mbps.com/fedora/linux/updates/20/x86_64/repodata/189bd24ec8b2062a45dbc5e01c213249a1f8be9a9f699195f832cc93b6394c0d-primary.sqlite.bz2: [Errno 14] HTTP Error 404 - Not Found Anderer Spiegelserver wird versucht. updates/20/x86_64/primary_db FAILED http://mirror2.hs-esslingen.de/fedora/linux/updates/20/x86_64/repodata/189bd24ec8b2062a45dbc5e01c213249a1f8be9a9f699195f832cc93b6394c0d-primary.sqlite.bz2: [Errno 14] HTTP Error 404 - Not Found Anderer Spiegelserver wird versucht. (5/8): logstash-1.4/primary_db | 17 kB 00:00 (6/8): fedora-chromium-stable/20/x86_64/primary_db | 29 kB 00:00 (7/8): updates/20/x86_64/primary_db | 12 MB 00:04 (8/8): fedora/20/x86_64/primary_db | 18 MB 00:04 (1/11): rpmfusion-free-updates/20/x86_64/group_gz | 1.6 kB 00:00 (2/11): rpmfusion-free-updates/20/x86_64/primary_db | 348 kB 00:00 (3/11): rpmfusion-nonfree/20/x86_64/group_gz | 1.0 kB 00:00 (4/11): rpmfusion-nonfree-updates/20/x86_64/group_gz | 1.0 kB 00:00 (5/11): rpmfusion-nonfree/20/x86_64/primary_db | 137 kB 00:00 (6/11): rpmfusion-nonfree-updates/20/x86_64/primary_db | 107 kB 00:00 updates/20/x86_64/updateinfo FAILED http://mirror2.hs-esslingen.de/fedora/linux/updates/20/x86_64/repodata/418dbd12af6795571f815b836aada093514abb1e7293fccfbed97c63ad70de16-updateinfo.xml.gz: [Errno 14] HTTP Error 404 - Not Found Anderer Spiegelserver wird versucht. (7/11): rpmfusion-free/20/x86_64/group_gz | 1.6 kB 00:00 updates/20/x86_64/pkgtags FAILED http://mirror.1000mbps.com/fedora/linux/updates/20/x86_64/repodata/3a810fcd709a53bec6270b989c4799367c19b85c0d35112cc0835ce7479e078b-pkgtags.sqlite.gz: [Errno 14] HTTP Error 404 - Not Found Anderer Spiegelserver wird versucht. (8/11): rpmfusion-free/20/x86_64/primary_db | 445 kB 00:00 updates/20/x86_64/updateinfo FAILED http://mirror.1000mbps.com/fedora/linux/updates/20/x86_64/repodata/418dbd12af6795571f815b836aada093514abb1e7293fccfbed97c63ad70de16-updateinfo.xml.gz: [Errno 14] HTTP Error 404 - Not Found Anderer Spiegelserver wird versucht. updates/20/x86_64/pkgtags FAILED http://mirror2.hs-esslingen.de/fedora/linux/updates/20/x86_64/repodata/3a810fcd709a53bec6270b989c4799367c19b85c0d35112cc0835ce7479e078b-pkgtags.sqlite.gz: [Errno 14] HTTP Error 404 - Not Found Anderer Spiegelserver wird versucht. (9/11): adobe-linux-x86_64/primary | 1.2 kB 00:00 (10/11): updates/20/x86_64/updateinfo | 1.4 MB 00:00 (11/11): updates/20/x86_64/pkgtags | 1.2 MB 00:00 Determining fastest mirrors * fedora: be.mirror.eurid.eu * rpmfusion-free: download1.rpmfusion.org * rpmfusion-free-updates: download1.rpmfusion.org * rpmfusion-nonfree: download1.rpmfusion.org * rpmfusion-nonfree-updates: download1.rpmfusion.org * updates: fedora-mirror01.rbc.ru adobe-linux-x86_64 2/2 No packages marked for update
So, yum tells me no update available.
[root@mysystem ~]# date ; dnf clean all Do 25. Sep 17:51:06 CEST 2014 Quellen werden aufgeräumt:logstash-1.4 mbooth-eclipse-luna : rpmfusion-free-updates fedora-HandBrake : rpmfusion-nonfree-updates rpmfusion-free : adobe-linux-x86_64 fedora rpmfusion-nonfree : fedora-chromium-stable updates Alles wird aufgeräumt [root@mysystem ~]# date ; dnf update Do 25. Sep 17:51:13 CEST 2014 logstash repository for 1.4.x packages 251 kB/s | 364 kB 00:01 The eclipse-luna Software Collection for Fedo 6.9 MB/s | 388 kB 00:00 RPM Fusion for Fedora 20 - Free - Updates 6.2 MB/s | 389 kB 00:00 Open source multiplatform video transcoder 58 kB/s | 6.5 kB 00:00 RPM Fusion for Fedora 20 - Nonfree - Updates 3.1 MB/s | 106 kB 00:00 RPM Fusion for Fedora 20 - Free 7.1 MB/s | 487 kB 00:00 Adobe Systems Incorporated 3.1 kB/s | 1.8 kB 00:00 Fedora 20 - x86_64 3.1 MB/s | 36 MB 00:11 RPM Fusion for Fedora 20 - Nonfree 5.9 MB/s | 289 kB 00:00 Builds of the "stable" tag of the Chromium We 82 kB/s | 241 kB 00:02 Fedora 20 - x86_64 - Updates 10 MB/s | 26 MB 00:02 Abhängigkeiten sind aufgelöst. .............. Transaktionsübersicht ============================================================================== Installieren 2 Packages Aktualisieren 80 Packages Entfernen 4 Packages
and dnf tells me my updates are ready to install! Guest I should start using dnf from now on.
After a lot of hard work I finished my bachelor thesis end of August and gave my colloquium end of September. Because of all the other stuff going on in my life I just have now time to upload my thesis.
I’m interested in log file analysis for a long time, but in the last years a lot has happened in this area. Here the abstract:
This thesis gives an overview on the Open Source and Free Software tools available for a centralized and structured log file analysis. This includes the tools to convert unstructured logs into structured log and different possibilities to transport this log to a central analyzing and storage station. The different storage and analyzing tools will be introduced, as well as the different web front ends to be used by the system administrator. At the end different tool chains will be introduced, that are well tested in this field.
Because of the time delay, some infos are already dated, but only two things, as fas as I know:
1. Graylog2 has now a new version available in Beta.
2. logstash has release a new version 1.2.1
I will keep this updated over time here on the blog, so have a look from time to time. If you find any problem, please inform me here at the blog or via mail: “logfiles jens.kuehnel.org” (add the @ at the right place).
The URL of the Thesis is at: http://www.kuehnel.org/bachelor.pdf
Have a log of fun and tell me what you think.
If you need to install virt-manager on a Fedora/Red Hat minimal installation, add the following packages:
yum install xorg-x11-fonts-Type1 xorg-x11-xauth libvirt-daemon-kvm libvirt virt-manager qemu-kvm qemu-kvm-tools
Hi,
because I don’t want to create a Google Account, here my comment to the blogpost Time on Computers, NTP, Certificates, etc from Ivan Makfinsky.
The simplest method is to put the following line into the pre script:
rdate SATELLITESERVER hwlock --systohc
On the Satellite this commands are necessary:
yum install -y xinetd chkconfig time-stream on chkconfig time-dgram on
I just got an new HP Microserver for a customer. I only had two 500GB disc available and installed Centos onto it. But now the 4* 3TB discs have arrived and I need to move everything from the 2 small discs to the new large discs.
Of course I could do a reboot, boot into a rescue CD and copy the data, but I don’t want to boot! Why no reboot? because I can! 🙂
I installed Centos onto the 2 disc with md0 as a 500MB RAID1 containing /boot, and md1 as a RADI1 containing the rest of the discs hosting a LVM Physical Volume.
This configuration is not guaranteed to work with every setup. Booting with a Bios from a GPT Partition should not work. It works on a HP Microserver, but it does not work on a Asus Motherboard I tried it as well. Of course as always: If you follow this setup and it breaks, eats you data, your homework or you cat. It is you own fault, don’t blame me!
1.) I started by stopping the RAID for sdb to remove this disc.
mdadm -f /dev/md0 /dev/sdb1 mdadm -r /dev/md0 /dev/sdb1 mdadm -f /dev/md1 /dev/sdb2 mdadm -r /dev/md1 /dev/sdb2
2. I removed the disc from the machine and put it into a USB/SATA Converter, and put it back into the RAID. Nowadays it’s very fast because the RAID detects what is still in sync. I feared a long wait to sync 500GB over USB, but is was done in seconds instead. Nice!
mdadm -a /dev/md0 /dev/sdb1 mdadm -a /dev/md1 /dev/sdb2
3. Next I removed the remaining disc from the RAID and remove it from the case. Now you have a backup disc in case something goes wrong now!
mdadm -f /dev/md0 /dev/sda1 mdadm -r /dev/md0 /dev/sda1 mdadm -f /dev/md1 /dev/sda2 mdadm -r /dev/md1 /dev/sda2
4. Now I plugged in the 4 new 3TB hard discs. I run the usual badblocks -v -v -w on it, before I installed it. Create on every disc 2 partitions and mark them as Linux SW RAID.
parted -s -- /dev/sda \ mklabel gpt \ mkpart boot-raid ext2 1M 525M \ toggle 1 raid \ mkpart lvm-raid ext2 525M -1 \ toggle 2 raid
5. Add the 500MB partitions to md0. Remove the old Partition from the USB-Disk and extend the RAID from a 2 disc RAID1 to a 4 disc RAID1.
mdadm -a /dev/md0 /dev/sda1 mdadm -a /dev/md0 /dev/sdc1 mdadm -a /dev/md0 /dev/sdd1 mdadm -a /dev/md0 /dev/sde1 mdadm -f /dev/md0 /dev/sdb1 mdadm -r /dev/md0 /dev/sdb1 mdadm -G -n 4 /dev/md0
6. Create a new RAID. Is use RAID 5 named /dev/md2 and create a Physical Volume on it.
mdadm -C -n 4 -l 5 /dev/md2 /dev/sda2 /dev/sdc2 /dev/sdd2 /dev/sde2 pvcreate /dev/md
7. Extend the existing Volume Group to /dev/md2 and move all Data from md1 to md2. Remove md1 from the Volume Group when done and destroy md1.
vgextend vg_name /dev/md2 pvmove /dev/md1 /dev/md2 vgreduce vg_name /dev/md1 mdadm -S /dev/md
8. The hardest stop is to make the boot possible. You need to get the UUID of the new RAID1 and add that to the grub.conf. Also you need to update your mdadm.conf and recreate your initramfs. Finally you need to install grub again onto the new sda.
mdadm -D /dev/md2 | grep UUID | sed -e 's/UUID : //' #add resulting UUID with rd_MD_UUID= to all kernels mdadm --examine --scan >> /etc/mdadm.conf dracut -f /boot/initramfs-$(uname -r).img $(uname -r) grub-install /dev/sda
9. reboot.
Wait, why reboot now, when I tried not to reboot? Because sooner or later you have to reboot and I want to now know if that will work.
I was shocked when I heard about Seth Vidal’s death. Of course I use yum daily, but it brought tears to my eyes, when I was reading my “my TODO List after a install” and realized that Seth was one of two people who responded. Thanks Seth, I will remember you.
So in reference to him, here my first ansible playbook:
---
- hosts: all
user: root
tasks:
- name: make sure eth0 starts at boot
lineinfile: dest=/etc/sysconfig/network-scripts/ifcfg-eth0 regexp=^ONBOOT= line=ONBOOT=yes backup=yes
- name: put ssh-key in
authorized_key: user=root key="{{lookup('file', '~/.ssh/id_dsa.pub') }}" manage_dir=yes
- name: get epel-repo rpm RHEL6
get_url: dest=/tmp/epel-release.rpm url=http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
when: ansible_os_family == 'RedHat' and ansible_lsb.major_release|int == 6
- name: get epel-repo rpm RHEL5
get_url: dest=/tmp/epel-release.rpm url=http://download.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm
when: ansible_os_family == 'RedHat' and ansible_lsb.major_release|int == 5
- name: install epel-repo rpm
yum: pkg=/tmp/epel-release.rpm state=installed
- name: install my packages
yum: pkg={{ item }} state=installed
when: ansible_os_family == 'RedHat' and ansible_lsb.major_release|int == 6
with_items:
# - mmv
- policycoreutils-python
- mod_ssl
- screen
- policycoreutils-python
- iotop
- yum-plugin-ps
- yum-cron
- iptraf
- acpid
- man
- bind-utils
- vim-enhanced
- nc
- zip
- unzip
- wget
- etckeeper
- links
- screen
- yum-utils
- lsof
- bash-completion
- ddrescue
- dos2unix
- dstat
- lftp
- links
- hdparm
- smartmontools
- jwhois
- kexec-tools
- mc
- mcelog
- memtest86+
- mtr
- nmap
- ntp
- openssh-server
- pbzip2
- rng-tools
- sysstat
- vconfig
- vlock
- lzop
- atop
- mosh
- name: activate autoupdate
service: enabled=yes state=started name=yum-cron
- name: initialize etckeeper
command: /usr/bin/etckeeper init creates=/etc/.git/description
- name: make first commit
command: /usr/bin/etckeeper commit -m "init" creates=/etc/.git/COMMIT_EDITMSG
I had to reinstall a couple of machines recently and I had to do the same thing more then once. So I wrote a script,and for my own future self I post it here, as reference.
sed -e 's:ONBOOT=no:ONBOOT=yes:' -i /etc/sysconfig/network-scripts/ifcfg-eth0 yum install http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-9.noarch.rpm http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-10.noarch.rpm yum update yum install policycoreutils-python iotop yum-plugin-ps yum-cron iptraf acpid man bind-utils vim-enhanced nc zip unzip wget etckeeper links screen yum-utils lsof bash-completion ddrescue dos2unix dstat lftp links hdparm smartmontools jwhois kexec-tools mc mcelog memtest86+ mmv mtr nmap ntp openssh-server pbzip2 rng-tools sysstat vconfig vlock lzop atop mosh mkdir /root/.ssh/ echo 'ssh-dss AAAAB3NzaC1kc3MAAACBAKk1vlmmXqEEeyrIfvhIXaFy7E8mu39/nXvJ1UVqtwLPJedSszGKtaRGPMw/0D+csOP61mPCNHCzX8EUbrv5DZ0OB1PFaPWC+IAsXcentCO3Ssy0syqNCYYGpumUK1ycsACJbO4oiwyPJTHe2BkI8laXDjRLdrbryPD79h8k9Kd7AAAAFQCWJNXBY3gGZA6lcXZYmaWUqaCTHQAAAIARzJIR6q+GD2nDKA11A0uKrjBnJ2HzBLb9KOr+Psj2jMEmAVosvpw+NIpaCf7yNOTTrl1oTC9ziopKTGe0emxWz00Zrwvu9gxxAa0eBBVzUuVD6fSmNSGTq1mfQASF0Qhwx6BIjgnxjTZVBvuTgCgR2Kk3/tLXDK2rXWC1MTUQNAAAAIAlq8y8nWOJlSWsEctWvNQkb5av0VQo00co3qUAtiGlICwVUpOleaz2c5r7v99JrqUpG/v5IlLBzc/w7Wa4UX10gW8sAurYfIx7LnrsqrprA+yQVwDcTSmolIMAacLgO9C4IWquJKfJiGmve+0OWJ6s9sDK2vip5GnZGN5NxEPw1Q== normal' >> /root/.ssh/authorized_keys chmod 700 ~/.ssh/ chmod 600 ~/.ssh/authorized_keys restorecon ~/.ssh/authorized_keys ~/.ssh/ etckeeper init etckeeper commit chkconfig yum-cron on
Before people complain! Yes, I know puppet and chef. At work I use it daily, but this are play machines and I don’t want to handle puppet here at home.
If you have a RHEL6.1 and try to update to RHEL6.4, you will get some strange errors, about missing libraries.
The reason is that ldconfig will not run anymore, because /etc/ld.so.conf.d/kernel-2.6.32-* sepcifies hwcap 0 till RHEL6.1 and RHEL 6.2 and higher specifies hwcap 1.
It the settings are different you get this error message:
ldconfig: /etc/ld.so.conf.d/kernel-2.6.32-71.29.1.el6.x86_64.conf:6: hwcap index 1 already defined as nosegneg
ldconfig cannot run after installing new libraries and some symlinks are broken.
To Fix: simply change all hwcap setting to 1 and rerun ldconfig.
After I thought I understood memory manager after all, this comes along. I have a behavior I can’t understand or explain. Let’s see if anyone can solve the puzzle.
You have a machine, lets say with 8GB Ram. You only run a small number of process on that machine and you want to add another service.
You run free -m and it looks like this:
total used free shared buffers cached Mem: 7980 4814 3165 0 1 4593 -/+ buffers/cache: 220 7759 Swap: 1759 0 1759
And you think hey, no problem a lot of free memory available. 220MB “real used” memory and 7759MB “freeable memory”. Let’s bring it on.
You start you application (I use memhog from numactl) which eats 4G of RAM and gives you a OOM.
Who knows why?
(The solution is attached as base64 encoded block) To get a peek simple execute the block in a shell.
cat << EOF | openssl base64 -d VGhlIHByb2JsZW0gbGllcyBpbiB0aGUgZmFjdCB0aGF0IHRlbXBmcyBpbiAvZGV2 L3NobSBpcyBmdWxsLgpXaHkgdGhlIGNvbnRlbnQgb2YgL2Rldi9zaG0vIGlzIGNv bnNpZGVyZWQgY2FjaGUgYW5kIG5vdCB1c2VkIGlzIGJleW9uZCBtZS4gQnV0IGl0 IGlzIHRoaXMgd2F5LgoK EOF
I would still love an explanation why it is that way.