Flatland

Once I read an introductory text on big bang (in portuguese) that explained big bang as an explosion of a space on four dimensions, a really weird concept for beginners. Great reading :)

And now I found the book that inspired many of the explanations :) Flatland, a Romance of Many Dimensions, an 1884 book written by Edwin Abbott Abbott.

You can read the plot on Wikipedia if you want to. It inspired lots of things, such as some animations.

(Also, I found this one page version very convenient.)

PS: Interestingly, the author of that introduction didn’t cited this book as an inspiration, citing instead a much later tv show. Can memetics explain that?

Posted in Food for Thought | Tagged , | Leave a comment

Slow netbooks: How to convert H.264 HD movies to lower quality (but faster) DivX

What if you want to play a HD movie on a netbook? I tried it on my acer 532h, and it’s lagging; mplayer complains about the machine being too slow. The problem maybe isn’t the high quality per se, but the modern, more space-efficient algorithms (that will make files hopefully smaller, but are also slower to decode).

The most interesting way seems to be to let mplayer lower the quality on the fly. A slashdot user tells how to do it:

mplayer -autosync 30 -vfm ffmpeg \
  -lavdopts lowres=1:fast:skiploopfilter=all file

But here I’m getting a segfault. There is also a library called VDPAU, and mplayer can be compiled to use it; not sure if it will help.

Anyway, here is the script I’m using right now:

#!/bin/sh

[ "$#" -lt 2 ] && { echo "usage: $0 <input> <output>"; exit; }

rate=1600

mencoder-mt "$1" -o "$2" \
    -ovc lavc \
    -oac mp3lame \
    -vf scale=800:-3 \
    -lavcopts vcodec=mpeg4:vbitrate=${rate}

It converts the file (In my case, it’s was a mp4 file, the video being encoded with H.264, and the audio with AAC) to an old-fashioned DivX AVI, with mp3 audio. Not sure if it yeilds the best results, but I will try and see.

I was encoding the audio to ogg vorbis, but it was getting out of sync. The problem might be the AVI file format. But all other attempts with -oac lavc was out of sync too, so I’m avoiding it.

I tried to do -oac copy, but it was unsuccessful. It’s also possible to store uncompressed audio with -oac pcm, but then the file will be larger (it’s slightly faster to encode. maybe faster to decode too, not sure). If it’s still slow, try lowering the video bitrate.

More links: how to convert video files, mplayer docs (see chapters 10 and 11).

PS: Just to say, it works great :) The decrease in size was smaller than I thought (I hoped a space gain of 50%, and got about 30%), but it nows runs fast! I might try another approach next time, because it was slow as hell to encode (about 10FPS).

Posted in Linux | Tagged , , , , , , , , | Leave a comment

Books on Consciousness

In the last post, I wrote about a book on the nature of music. It was written by a computer science guy called Philip Dorrell, and can be read online. He wrote things on consciousness too.

He says:

Consciousness is a system for deciding whether or not to do the thing that you were going to do next.

And he goes as far as theorizing that this task can be done by a single neuron (or a group of redundant neurons that acts as a single neuron). He also makes a similar assumption on his book about music, too. Nice reading, I will probably comment more later.

He wrote on dreams too, on his page (and free will, and other things)

The Inner Light Theory of Consciousness

There is also a book called The Inner Light Theory of Consciousness, written by Steven W. Smith. He is also a computer guy, and the book can be read online.

He starts explaining the dichotomy between brain and mind, and some traditional philosophical views. He then talks about information processing, to meet the conclusion that many brain researches have: the brain runs simulations of the environment, and our consciousness perceive those simulations, not actual reality (The thought-provoking cover of the book explores this fact. The cover alone would make me to buy this book, if I ever encountered it :)).

The novelty of the book is to consider this to be the very reason for our perception of the mind/body problem: we can see the brain of others as they actually are (a bunch of microscopic neurons, or macroscopically an amorphous mass), but we can’t see how our own brain works while we think. Without this, he says, the hard difference between mind and body wouldn’t exist. The easy analogy is to consider we live in a matrix, created by our own brain, but the one he uses is an episode of star Trek called the inner light, from where he borrows the title.

He then starts to imagine how would be a being that actually had full understanding on every bit of every part of her neurvous system. His conclusion is that this being would not recognize consciousness (or free well, or many other things) as indivisible, fundamental concepts.

His final conclusion is quite surprising: consciousness is not a characteristic you need to add to a soulless android that behaves like human, so that he could have a mind. Conscience is something you need to take from it. Worth a read :)

(Smith has also put on the internet The Scientist and Engineer’s Guide to Digital Signal Processing, a newbie-ready book on digital signal processing. It made me grasp some things such as JPEG compression. And I must cite he worked on evil tools for evil empires ®, and also on some medical systems, so he must know about a bit about signal processing :) BTW since I have no bookmark I will put another good quality link on fourier theory)

(may I also post something on lucid dreams, in special on how to remember your dreams? also see wikipedia)

Posted in Food for Thought | Tagged , , , , , , , | 2 Comments

Air – a computer-generated music (using Haskore)

Very nice thingy, a music generated by a program. Sounds like SNES music for me :~

Air was programmed using Haskore, a Haskell library for manipulating music, and used SuperCollider to output audio.

Here is the source code. I see it first defined basic melodies, and transformed them, so maybe it doesn’t qualify as 100% computer-generated; but I wonder if this thing actually exists. Wikipedia has an article on algorithmic composition anyway. (The obligatory joke here is the serious research attempts to prove that pop music is so repetitive that it can be fully automated by a computer :))

Also on this subject (and a lot more) is the book what is music by Philip Dorrell. Insightful! and it’s available for reading for free, I recommend :)

ps: there is a youtube link too (I desisted to embed here because the audio is crappy, but you can see some fractals anyway)

Posted in Haskell | Tagged , , , | Leave a comment

Ruby enumerators (and a script to list all Google search results)

A while ago I carefully crafted a google searcher in sed and shell script. Making scripts for doing http requests is cool! (I say carefully because it is a pain to parse HTML with regular expressions :))

The Code

Nokogiri is a ruby library for searching HTML, but a very simple one. On the start page they have a simple script that searches google. It’s beautiful.

But it doesn’t search the whole results, just the first page. So I modified it a bit:

#!/usr/bin/env ruby

# http://iamstealingideas.wordpress.com/2010/05/23/ruby-enumerators-and-ascript-to-list-all-google-search-results/

require 'open-uri'
require 'nokogiri'

class Search
  include Enumerable

  def initialize(terms)
    def escape u
      URI.escape u, Regexp.new("[^#{URI::PATTERN::UNRESERVED}]")
    end

    @terms = terms.map { |u| escape u }.reduce { |a, b| "#{a}+#{b}" }
  end

  def each
    10.times do |n|
      url = "http://www.google.com/search?start=#{100*n}&num=100&q=#{@terms}"

      doc = Nokogiri::HTML(open(url))
      break if n > doc.css('table#nav a.fl').length
      doc.css('h3.r a.l').each { |p| yield p['href'] }
    end
  end
end

fail "Usage: #{$0} terms" if $*.empty?

Search.new($*).each { |p| puts p }

It was tested in Ruby 1.9.

It’s executable: save it and try ./script site:iamstealingideas.wordpress.com

Objects of Search class has the each method, that receives a block and calls it for all search results. (In this case, the block just prints the url, with puts).

But it also has other methods. Let’s suppose your terminal is being flooded with too much results. (Maybe I’m being too optimistic). If you want just the first 7 terms, you may use the first method. Replace the last line of the script with:

Search.new($*).first(7).each { |p| puts p }

(It may make no difference). What about counting the available results?

puts Search.new($*).count

Enumerable classes

The trick here is to include the Enumerable mixin. This will give a lot of methods to the class, all based on your each method. Just like, say, an Array:

[1, 2, 3].each { |p| puts p }
puts [1, 2, 3].count

In fact, it has all this methods, but it doesn’t have, say, []. You may convert it to the class Array using entries. Replacing the last line with:

puts Search.new($*).entries[723]

You will have the 723th result. Yeah, you might try a keyword with more results, like ./search love.

More

If you have trouble understanding the code, you may want to read about blocks and methods, mixins (more on mixins), enumerators. Or asking in the comments :)

IMO, Ruby seems like a nice substitute to shell script. :)

Some random things:

  • Another way is to subclass the Enumerator class. Mixins seems to have higher precedence at method lookup, so you may prefer it if you want to implement some of those methods yourself.
  • That escape code was found there.
  • Stopping the search was tricky. I’m counting the number of pages.
  • &safe=off&filter=0 is probably useful. Also see this google help.
  • Map and reduce is a used a lot in functional programming. (Map is also known as collect; reduce, as inject or fold). Do you think this code is readable?
Posted in Ruby | Tagged , , , , , , | Leave a comment

Backups on an encrypted external hard drive

Here I assume you already filled it with random data.

Just one more post documenting my system.. (It might be useful only to myself, who knows).

I have a file /root/key with random numbers (just dd if=/dev/random of=/dev/sdb bs=key_size count=1).

It’s my key file. Losing this file, I lose the data on the external hide drive. The drive is on /dev/sdb. I just did:

cryptsetup luksFormat /dev/sdb /root/key
cryptsetup luksOpen /dev/sdb vault-metal -d /root/key
pvcreate /dev/mapper/vault-metal
vgcreate metal /dev/mapper/vault-metal
lvcreate -L 100G -n home metal
mke2fs -t ext4 /dev/metal/home

If you want to, you may use a passphrase, instead of a key file. Just omit /root/key from the first command, and -d /root/key from the second.

Anyway, now I have /dev/metal/home for mirroring my /home. I want to do backup with rsync, but, since I am not always with the external hd connected, I want it to not try to mount it before I plug it on USB. In fact I don’t want it to mount automatically at all.

I don’t know if /etc/crypttab can help me here, but a shell script will suffice:

#!/bin/bash

if
  cryptsetup luksOpen /dev/sdb vault-metal -d /root/key/metal &&
  vgscan &&
  vgchange -a y metal &&
  mount /dev/metal/home /metal/home
then
  rsync -avh --delete /home/ /metal/home
else
  echo Error\?
fi

I just run it for mounting. (I might need to change it for checking for UUID. Actually, fstab would help here). One for umounting it would be good, too:

#!/bin/bash

if
  umount /metal/home &&
  vgchange -a n metal &&
  cryptsetup luksClose /dev/mapper/vault-metal
then
  echo Ok
else
  echo Error\?
fi

There is some info on rsync here on this wiki, and on the gentoo wiki too. If you are serious about backups you should check the documentation too.

There is some info (and links) on dm-crypt/LUKS and LVM on an earlier post, too.

Also this is the first time ever I use rsync :)

Posted in Linux | Tagged , , , , | Leave a comment

TPB CANT BE SHUT DOWN

Yeah, I know they are no saints and just took to themselves the money donated to buy Sealand. They are selling the business too, etc, etc, etc.

It’s irrational, but I just love TPB. :cry:

Posted in Internets | Tagged , , , , , | Leave a comment

How to mix Debian testing and unstable

Like me, you might want to use debian testing but occasionally install unstable packages. What happens if you simply put testing and unstable repositories on your sources.list?

If you try that, you will notice that your system will always prefer to install packages from unstable (because apt loves newer versions!). You should, then, put this line on your /etc/apt/apt.conf:

APT::Default-Release "testing";

Initially I thought this feature were called pinning, but pinning gives finer, per package control (see links below).

In order to install something from unstable, you must override the default:

aptitude -t sid install some-package

But, when the version of this package on testing pass unstable, it will install from testing again. (This also works with stable/testing; just make sure you default to stable)

Note that AptPreferences on debian wiki will warn against package breakage (but you already knew it, right?). There is a comprehensive tutorial on mixing stable and testing on the wiki of a guy named Freek Dijkstra, also.

Isn’t Debian great? :)

Posted in Linux | Tagged , , | 1 Comment

Script to write random data to a hard drive using openssl

The script I posted on how to write random data to the hard drive was kinda buggy..

Here is a better version, that actually stop when it finishes. It also shows the progress and does some safety checking (It’s weird, but I actually don’t enjoy much of this. I like the feeling of having to be a lot careful at the console. Sorry, can’t explain)

I also removed a lot of quotes I found unnecessary (a thing that surely would chock the folks at #bash @ irc.freenode.org). So I bet there is some bug hidden that I couldn’t spot. Use at your own risk :)

#!/bin/bash

# The author of this script, Elias Amaral,
# claims no copyright over it.

# http://iamstealingideas.wordpress.com/2010/05/20/writing-random-data-to-a-hard-drive-again

msg() {
  printf "\n - $1\n\n" $2
}

mbs=4 # 4mb
blocksize=$(($mbs * 1024 * 1024))

dev=$1

if [[ -z $dev ]]; then
  msg "usage: $0 <device>"; exit
elif [[ ! -b $dev ]]; then
  msg "$dev: not a block device"; exit
elif [[ ! -w $dev ]]; then
  msg "$dev: no write permission"; exit
elif grep -q $dev /etc/mtab; then
  msg "$dev: mounted filesystem on device, omgomg!"; exit
fi

cat <<end
This program writes random data to a hard disk.
It is intended to be used before storing encrypted data.
It may contain bugs (but seems to work for me).

It seems you have chosen to wipe data from the disk $dev.
Here is the partition table of this disk:
end

fdisk -l $dev

echo
echo 'Are you sure you want to proceed?'

msg 'WARNING: IT WILL DESTROY ALL DATA ON THE DISK'

read -p 'Type uppercase yes if you want to proceed: ' q

if [[ $q != YES  ]]; then
  exit
fi

while
  echo $i > step.new
  mv step.new step

  msg 'Writing at offset %s' $(($mbs * $i))M

  openssl rand \
          -rand /dev/urandom \
          $blocksize | \

  dd of=$dev \
     bs=$blocksize \
     seek=$i
do
  let i++
done

msg Finished.
Posted in Linux | Tagged , , , , | 1 Comment

Running your own dns cache on Debian (with djbdns)

Run as root:

aptitude install djbdns dnscache-run
mv /etc/resolv.conf /etc/resolv.conf.orig
echo nameserver 127.0.0.1 > /etc/resolv.conf

Then configure your connection to not update /etc/resolv.conf upon each connection (I use pppoe to connect to adsl, so I needed to comment the line “usepeerdns” on /etc/ppp/peers/dsl-provider)

Straightforward, isn’t it? For other operating systems, see the official tutorial.

Posted in Linux | Tagged , , , | Leave a comment