Saturday, August 8, 2009

LANG vs LC_MESSAGES in man(1) error messages

At first I'd like to say that I'm pretty happy with english being my system's language. But since I have to use russian sometimes I've set LANG=ru_RU.UTF-8. Unfortunately man(1) error messages are encoded in KOI8-R so they look like garbage on my locale. I don't particularly want them to be in russian so I've set LC_MESSAGE=C. The man page locale(1P) says:

LC_MESSAGES
Determine the locale that should be used to affect the format
and contents of diagnostic messages written to standard error.

But it doesn't work for man(1). After little investigation I've learned that man(1) uses catgets(3) to get an error string. The catalog descriptor for catgets(3) is received from catopen(3) which (in case $LC_MESSAGES=C) carefully tries to open /usr/share/locale/C and some other dirs with C locale and finally fallbacks to /usr/share/locale/ru. If man(1) obtains an empty string from catgets(3) it looks for it in builtin table which is what I need.
As for now I can't say which part of this complicated situation is buggy, so I've inserted a hack into man(1) to get an error string right from builtin table if the locale is C or POSIX.
Here is my patch.

diff -rNu a/man-1.6f/src/gripes.c b/man-1.6f/src/gripes.c
--- a/man-1.6f/src/gripes.c 2006-11-21 22:53:44.000000000 +0300
+++ b/man-1.6f/src/gripes.c 2009-08-11 10:36:08.000000000 +0400
@@ -99,15 +99,22 @@
static char *
getmsg (int n) {
char *s = "";
-
- catinit ();
- if (catfd != (nl_catd) -1) {
- s = catgets(catfd, 1, n, "");
- if (*s && is_suspect(s))
- s = "";
- }
- if (*s == 0 && 0 < n && n <= MAXMSG)
- s = msg[n];
+ char *lm;
+
+ lm = getenv("LC_MESSAGES");
+ if (lm && (!strcmp(lm, "C") || !strcmp(lm, "POSIX"))) {
+ if (0 < n && n <= MAXMSG)
+ s = msg[n];
+ } else {
+ catinit ();
+ if (catfd != (nl_catd) -1) {
+ s = catgets(catfd, 1, n, "");
+ if (*s && is_suspect(s))
+ s = "";
+ }
+ if (*s == 0 && 0 < n && n <= MAXMSG)
+ s = msg[n];
+ }
if (*s == 0) {
fprintf(stderr,
"man: internal error - cannot find message %d\n", n);

Tuesday, February 24, 2009

IRC DCC auto leacher

Long ago in 2007 I wrote an irssi script to leach many packs from xdcc bots automaticaly. It's quite unfinished (ugly ui, etc) but I dind't touch it for ages although i'm still using it as it is. It supports downloading from only one bot at a time. If downloading breaks it will retry after timeout. Note that some bots hate if you retry right after a failure and they will ban you (sometimes waiting for more than 5 minutes is required).

Interfase is like this:

/xdccget queue [botname] [packs...] - you should start with this. After a minute xdccget will start leaching.
/xdccget add [packs...] - add more packs
/xdccget del - delete last pack
/xdccget pause - stop downloading, remove timers
/xdccget resume - resume downloading

here is the script:

# 2007, savrus
#
# Advanced downloader from xdcc bot. Independent on bot's language
#
# Based on XDCCget by Stefan "tommie" Tomanek
#
# Dustributed under GNU GPL v2 or higher

use vars qw($VERSION %IRSSI);
$VERSION = "20070321";
%IRSSI = (
authors => "savrus",
contact => "go to hell",
name => "xdccget",
description => "auto download from xdcc bot",
license => "GPLv2",
changed => "$VERSION",
commands => "xdccget"
);

use Irssi;
use vars qw(@g_queue $g_nick $g_server $g_witem $g_timer);

sub show_help() {
my $help="xdccget $VERSION
/xdccget queue Nickname ...
Queue the specified packs of the server 'Nickname'
/xdccget help
Display this help
";
print CLIENTCRAP $help;
}

sub sig_dcc_closed {
my ($dcc) = @_;
my ($dir,$file) = $dcc->{file} =~ m,(.*)/(.*),;

return unless $dcc->{type} eq 'GET';
return unless $dcc->{nick} eq $g_nick;

if ($dcc->{transfd} < $dcc->{size}) {
process_queue($dcc->{nick});
}
else {
# rename $dcc->{file}, "/ircdone/$file";
shift_queue($dcc->{nick});
process_queue($dcc->{nick});
}
}

sub cmd_xdccget {
my ($args, $server, $witem) = @_;
my @arg = split(/ /, $args);

if ($arg[0] eq 'queue'){
shift @arg;
initialize_queue("@arg", $server, $witem);
}
elsif ($arg[0] eq 'add'){
shift @arg;
add_to_queue(@arg);
}
elsif ($arg[0] eq 'del'){
dell_from_queue();
}
elsif ($arg[0] eq 'pause'){
pause();
}
elsif ($arg[0] eq 'resume'){
resume();
}
elsif ($arg[0] eq 'help') {
show_help();
}
else {
print CLIENTCRAP "xdccget: $g_nick @g_queue";
}
}

sub pause {
$nick = $g_nick;
clean();
$g_nick = $nick;
}

sub resume {
process_queue();
}

sub shift_queue {
shift @g_queue;
}

sub clean {
# Clean previous job
if ($g_nick) {
my $nick = $g_nick;
$g_server->command("MSG $g_nick xdcc remove");
$g_nick = "";
$g_server->command("DCC close get $nick");
}
if ($g_timer) {
Irssi::timeout_remove($g_timer);
}
}

sub initialize_queue {
my ($args, $server, $witem) = @_;
my @args = split(/ /, $args);

clean();

$g_nick = $args[0];
shift @args;
@g_queue = @args;
$g_server = $server;
$g_witem = $witem;

process_queue();
}

sub process_queue {
while ((@g_queue[0] <= 0) && (scalar @g_queue > 0)) {
shift @g_queue;
}
if (scalar @g_queue > 0) {
$g_timer=Irssi::timeout_add(60 * 1000, 'transfer', undef);
}
}

sub transfer {
Irssi::timeout_remove($g_timer);
$g_server->command("MSG $g_nick xdcc send @g_queue[0]");
}

sub add_to_queue {
my (@args) = @_;
push @g_queue, @args;
}

sub dell_from_queue {
pop @g_queue;
}

Irssi::signal_add('dcc closed', 'sig_dcc_closed');
#
# TODO: handle rejoining the channel (self disconnection + auto reconnection)
# Be careful: content may be changed in such situation.
#
#Irssi::signal_add('channel joined', 'sig_channel_joined');


## Used in a modified script for a bot that was always disconnecting from the channel.
## But be careful - usually this happens when bot owner shuffles the packlist
#
#sub sig_message_join {
# my ($server, $channel, $nick, $addr) = @_;
# if ($nick eq $g_nick){
# print CLIENTCRAP "XDCCGET DEBUG: $nick joined channel (wait for $g_nick)";
# $g_server->command("MSG $g_nick xdcc remove");
# $g_server->command("DCC close get $g_nick");
# if ($g_timer) {
# Irssi::timeout_remove($g_timer);
# }
# resume();
# }
#}
#Irssi::signal_add_last('message join', 'sig_message_join');


Irssi::command_bind('xdccget', \&cmd_xdccget);

print CLIENTCRAP '%B>>%n '.$IRSSI{name}.' '.$VERSION.' loaded';

Monday, February 23, 2009

mplayer screenshot name

About a year ago a friend of mine complained of the filename style for screenshots by mplayer. He said it should contain the original file name and a timestamp (so one could easily watch screenshoted scene once again). I hacked mplayer but unfortunately the patch didn't get accepted to the mainline.

Here is the patch anyway:

Index: libmpcodecs/vf_screenshot.c
===================================================================
--- libmpcodecs/vf_screenshot.c (revision 28346)
+++ libmpcodecs/vf_screenshot.c (working copy)
@@ -22,9 +22,19 @@
#include "libswscale/swscale.h"
#include "libavcodec/avcodec.h"

+#include "m_option.h"
+#include "m_struct.h"
+
+#include "mplayer.h"
+
+#define SHOT_FNAME_LENGTH 102
+#define TIMESTAMP_LENGTH 20
+
struct vf_priv_s {
int frameno;
- char fname[102];
+ char *fname;
+ char *basename;
+ int style;
/// shot stores current screenshot mode:
/// 0: don't take screenshots
/// 1: take single screenshot, reset to 0 afterwards
@@ -36,7 +46,7 @@
AVCodecContext *avctx;
uint8_t *outbuffer;
int outbuffer_size;
-};
+} vf_priv_dflt;

//===========================================================================//

@@ -92,14 +102,65 @@
else return 0;
}

-static void gen_fname(struct vf_priv_s* priv)
+static void gen_fname(struct vf_priv_s* priv, double pts)
{
- do {
- snprintf (priv->fname, 100, "shot%04d.png", ++priv->frameno);
- } while (fexists(priv->fname) && priv->frameno < 100000);
- if (fexists(priv->fname)) {
- priv->fname[0] = '\0';
- return;
+ char *base = "shot";
+ char tstamp[TIMESTAMP_LENGTH];
+ char ts_sep;
+ uint32_t hour = (uint32_t) (pts/3600);
+ uint32_t min = (uint32_t) (pts/60) % 60;
+ uint32_t sec = (uint32_t) pts % 60;
+ uint32_t msec = (uint32_t) (pts*100) % 100;
+ size_t n;
+
+ switch (priv->style) {
+ case 1:
+ case 2:
+ ts_sep = (priv->style == 1) ? ':' : '-';
+ /* TIMESTAMP_LENGTH is enough for 2^32 seconds */
+ snprintf(tstamp, TIMESTAMP_LENGTH,
+ "%02" PRIu32 "%c%02" PRIu32 "%c%02" PRIu32 ".%02" PRIu32,
+ hour, ts_sep, min, ts_sep, sec, msec);
+
+ if (priv->basename)
+ base = priv->basename;
+ else {
+ if (strstr(filename, "://"))
+ base = "shot";
+ else {
+ base = strrchr(filename, '/');
+ if (base == NULL)
+ base = filename;
+ else
+ base++;
+ }
+ }
+
+ /* 8 is length of ".[].png" plus '\0' */
+ n = strlen(base) + strlen(tstamp) + 8;
+ priv->fname = malloc(n);
+ if (!priv->fname) {
+ mp_msg(MSGT_VFILTER,MSGL_ERR,"Unable to allocate memory in vf_screenshot.c\n");
+ return;
+ }
+ snprintf(priv->fname, n, "%s.[%s].png", base, tstamp);
+ break;
+ default:
+ case 0:
+ priv->fname = malloc(SHOT_FNAME_LENGTH);
+ if (!priv->fname) {
+ mp_msg(MSGT_VFILTER,MSGL_ERR,"Unable to allocate memory in vf_screenshot.c\n");
+ return;
+ }
+ do {
+ snprintf (priv->fname, SHOT_FNAME_LENGTH, "shot%04d.png", ++priv->frameno);
+ } while (fexists(priv->fname) && priv->frameno < 100000);
+ if (fexists(priv->fname)) {
+ free(priv->fname);
+ priv->fname = NULL;
+ return;
+ }
+ break;
}

mp_msg(MSGT_VFILTER,MSGL_INFO,"*** screenshot '%s' ***\n",priv->fname);
@@ -196,11 +257,13 @@
if(vf->priv->shot) {
if (vf->priv->shot==1)
vf->priv->shot=0;
- gen_fname(vf->priv);
- if (vf->priv->fname[0]) {
+ gen_fname(vf->priv, pts);
+ if (vf->priv->fname) {
if (!vf->priv->store_slices)
scale_image(vf->priv, dmpi);
write_png(vf->priv);
+ free(vf->priv->fname);
+ vf->priv->fname = NULL;
}
vf->priv->store_slices = 0;
}
@@ -263,6 +326,8 @@
av_freep(&vf->priv->avctx);
if(vf->priv->ctx) sws_freeContext(vf->priv->ctx);
if (vf->priv->buffer) free(vf->priv->buffer);
+ if (vf->priv->fname) free(vf->priv->fname);
+ if (vf->priv->basename) free(vf->priv->basename);
free(vf->priv->outbuffer);
free(vf->priv);
}
@@ -278,7 +343,7 @@
vf->draw_slice=draw_slice;
vf->get_image=get_image;
vf->uninit=uninit;
- vf->priv=malloc(sizeof(struct vf_priv_s));
+ if (!vf->priv) vf->priv = calloc(1, sizeof(struct vf_priv_s));
vf->priv->frameno=0;
vf->priv->shot=0;
vf->priv->store_slices=0;
@@ -294,14 +359,27 @@
return 1;
}

+#define ST_OFF(f) M_ST_OFF(struct vf_priv_s,f)
+static const m_option_t vf_opts_fields[] = {
+ {"style", ST_OFF(style), CONF_TYPE_INT, 0, 0, 2, NULL},
+ {"basename", ST_OFF(basename), CONF_TYPE_STRING, 0, M_OPT_MIN, M_OPT_MAX, NULL},
+ {NULL, NULL, 0, 0, 0, 0, NULL}
+};

+static const m_struct_t vf_opts = {
+ "screenshot",
+ sizeof(struct vf_priv_s),
+ &vf_priv_dflt,
+ vf_opts_fields
+};
+
const vf_info_t vf_info_screenshot = {
"screenshot to file",
"screenshot",
"A'rpi, Jindrich Makovicka",
"",
screenshot_open,
- NULL
+ &vf_opts
};

//===========================================================================//
Index: DOCS/man/en/mplayer.1
===================================================================
--- DOCS/man/en/mplayer.1 (revision 28346)
+++ DOCS/man/en/mplayer.1 (working copy)
@@ -7180,15 +7180,28 @@
.RE
.
.TP
-.B screenshot
+.B screenshot[=style:basename]
Allows acquiring screenshots of the movie using slave mode
commands that can be bound to keypresses.
See the slave mode documentation and the INTERACTIVE CONTROL
section for details.
-Files named 'shotNNNN.png' will be saved in the working directory,
-using the first available number \- no files will be overwritten.
The filter has no overhead when not used and accepts an arbitrary
colorspace, so it is safe to add it to the configuration file.
+.RSs
+.IPs <style>
+0: Files named 'shotNNNN.png' will be saved in the working directory,
+using the first available number \- no files will be overwritten.
+.br
+1: Files named 'basename.[hh:mm:ss.ms].png' will be saved in the
+working directory.
+.br
+2: Files named 'basename.[hh-mm-ss.ms].png' will be saved in the
+working directory. Use this if your environment doesn't allow colon
+in filenames.
+.IPs <basename>
+Basename for a screenshot file. If not set and mplayer is playing a file,
+the file name will be used. If not set and mplayer is playing a stream,
+"shot" will be used.
.RE
.
.TP
Index: mencoder.c
===================================================================
--- mencoder.c (revision 28346)
+++ mencoder.c (working copy)
@@ -115,6 +115,7 @@
char* audio_lang=NULL;
char* dvdsub_lang=NULL;
static char* spudec_ifo=NULL;
+char* filename=NULL;

static char** audio_codec_list=NULL; // override audio codec
static char** video_codec_list=NULL; // override video codec
@@ -397,7 +398,6 @@
double v_timer_corr=0;

m_entry_t* filelist = NULL;
-char* filename=NULL;

int decoded_frameno=0;
int next_frameno=-1;

Tuesday, November 18, 2008

pam_setquota, pam_kill

Being network administrator at MSU dorm two years ago, I made a public ssh server. Users were presented in mysql database on another server, so I used pam_mysql and libnss_mysql, which were already existed, for my public ssh server. I also wanted to set disk quota automatically for each user, but linux setquota(8) doesn't allow you to edit quota for non-existing user. Nor did work any pam_setquotas I found. So I wrote a one myself.

Of course, I edited limits.conf. But it didn't save from stupid cpu-intensive "while(1);" programs some nasty users had left running. I decided to kill every user's process, if he/she is no longer logged on the system, and wrote pam_kill for that.

Today, to share my old code, I created two Google Code projects: pam-setquota and pam-kill. You can access them via my Google Code profile.

Saturday, November 15, 2008

crc32

Long time passed since my last play with crc32. I wanted to learn and benchmark zlib's implementation as well (it's quite complex in comparison to basic ones), but it seems it will stuck in my todo list for ages. So, I decided to write what I know right now.

This is a very simple implementation taken from rhash (rhash.sourceforge.net). Every article about crc32 describes the code below, it is not something outstanding from rhash.
unsigned get_crc32(unsigned crcinit, const char *p, int len) {
register unsigned crc;
const char *e = p + len;

for(crc=crcinit^0xFFFFFFFF; p<e; p++)
crc = crcTable[(crc^ *p) & 0xFF] ^ (crc >> 8);
return( crc^0xFFFFFFFF );
}

And this is an x86 asm code, produced by gcc 4.1.2 from the 'for' loop:
.L11:
movsbl (%ecx,%esi),%eax
incl %ecx
xorl %edx, %eax
andl $255, %eax
shrl $8, %edx
xorl crcTable(,%eax,4), %edx
cmpl %ebx, %ecx
jne .L11

One of my friends found a bit faster implementation written in inline asm and used it for his hash checker ArXSum. I will not give here his code, because my optimization of rhash code is even faster. ArXSum just gave me an idea to use 8-bit registers to get rid of the andl instruction.
First, I enforced gcc to use 8-bit register. I hoped it would be enough, but it won't.
unsigned get_crc32(unsigned crcinit, const char *p, int len) {
register unsigned crc;
unsigned char m;
const char *e = p + len;
m = 0;

for(crc=crcinit^0xFFFFFFFF; p<e; p++) {
m = (crc^ *p);
crc = crcTable[m] ^ (crc >> 8);
}
return( crc^0xFFFFFFFF );
}

The code produced is
.L11:
movzbl (%ecx,%esi), %eax
incl %ecx
xorb %dl, %al
movzbl %al, %eax
shrl $8, %edx
xorl crcTable(,%eax,4), %edx
cmpl %ebx, %ecx
jne .L11

Do you see that utterly useless second movzbl? My final optimization was just to remove it and to add "xorl %eax,%eax" before the loop (that would be "m = 0" which had been lost by gcc). Newest version of gcc also produces the same code.

I still want to carefully look into zlib one day and to compare their high-level optimization with mine. I will eventually post about it.

Monday, October 29, 2007

gcc optimizer

I found that the following code causes segfault when compiled with -O2. Compiler:
gcc (GCC) 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)

#include <stdlib.h>

struct prime {
struct prime *next;
};

int main(){
struct prime primes = {.next=NULL};
struct prime *p = ℙ

while (p->next != NULL){
p = p->next;
}
p->next = (struct prime*) malloc (sizeof(struct prime));
return 0;
}

Sunday, October 14, 2007

I've just started a new blog. I'll try to keep it clean and for my own accomplishments only.