Things To Remember

Making Stuff Faster -- MySQL to be Specific · 134 days ago by Dylan Doxey

I don't have much patience for slow running programs. Heck, I could type the data into the database faster than this one particular program.

Sounds like an opportunity to employ the slow query log feature. It's easy enough to enable. Just open up /etc/mysql/my.cfn and uncomment the relevant lines.

# Here you can see queries with especially long duration
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time  = 2

Once you've accomplished that, just tail the query log while you program runs and become enlightened.

$: tail -F /var/log/mysql/mysql-slow.log

# Time: 110915 14:35:41
# User@Host: url_urs[url_usr] @ localhost []
# Query_time: 375.707403  Lock_time: 0.000260 Rows_sent: 1  Rows_examined: 1200487
SET timestamp=1316122541;
SELECT url, url_id FROM url WHERE visit_date = 0 ORDER BY RAND() LIMIT 1;

Well, there's the culprit. If the query only wants one result, then obviously there's no need to examine 1.2 million records to find it.

Fortunately, there's a convenient way to inform the dataserver about this obviousness.

$: mysql url_db -p -u url_usr

mysql> CREATE INDEX visit_date_index ON url(visit_date);
Query OK, 614526 rows affected (5 min 31.02 sec)
Records: 614526  Duplicates: 0  Warnings: 0

That should do it.

Unquoting Your Hash Keys · 383 days ago by Dylan Doxey

I once thought it was pretty neat to format my hash keys like this.

1 use strict;
2 use warnings;
3 
4 my $day   = $date_rh->{ 'day'   };
5 my $month = $date_rh->{ 'month' };
6 my $year  = $date_rh->{ 'year'  };
7

I have now flipped from that flop.

Here's how I like to see my keys formatted now.

1 use strict;
2 use warnings;
3 
4 my $day   = $date_rh->{day};
5 my $month = $date_rh->{month};
6 my $year  = $date_rh->{year};
7

Now whenever I open up a program and start making tweaks one of the first things to go is the old line bloating style of hash keys.

However, this is an excersize in tedium when one is not adept with vim pattern matching.
So here's a tip.

:%s/{[ ]*'\([A-Za-z0-9_]\+\)'[ ]*}/{\1}/g

The key with this pattern is knowing what to escape with the backslash. Here you can see that the pattern meta-characters are backslashed.

Enjoy.

Ubuntu Release Upgrade · 448 days ago by Dylan Doxey

The first time I did a release upgrade on my Ubuntu desktop machine I just did a substitution replacement in my /etc/apt/sources.list (something) like this:

$: sudo perl -pi -e 's{dapper}{hardy}g' /etc/apt/sources.list

And then followed by:

$: sudo apt-get update
...
$: sudo apt-get upgrade
...

That approach worked, that time. The next release cycle I advised a coworker to do the same, and that was a disaster!

From that time forward I was careful to always use the upgrade manager GUI tool to do my release upgrades. This has been easy and relatively disaster free.

But what about my server?

I came to the conclusion recently that doing apt-get dist-upgrade wasn't getting me the release upgrades and I was in danger of running on a relase I couldn't upgrade from.

Note to self: if you want to do a release upgrade, use the do-release-upgrade tool.

$: sudo do-release-upgrade

I shall now live happily ever after.

Roku BrightScript Syntax Highlighting in Vim · 496 days ago by Dylan Doxey

I don't know how everyone else is editing their BrightScript code, but I want to use vim.

  1 ' ********************************************************************
  2 ' **  Sample PlayVideo App
  3 ' **  Copyright (c) 2009 Roku Inc. All Rights Reserved.
  4 ' ********************************************************************
  5 
  6 Sub Main()
  7     'initialize theme attributes like titles, logos and overhang color
  8     initTheme()
  9 
 10     'display a fake screen while the real one initializes. this screen
 11     'has to live for the duration of the whole app to prevent flashing
 12     'back to the roku home screen.
 13     screenFacade = CreateObject("roPosterScreen")
 14     screenFacade.show()
 15 
 16     itemMpeg4 = {   ContentType:"episode"
 17                SDPosterUrl:"file://pkg:/images/DanGilbert.jpg"
 18                HDPosterUrl:"file://pkg:/images/DanGilbert.jpg"
 19                IsHD:False
 20                HDBranded:False
 21                ShortDescriptionLine1:"Dan Gilbert asks, Why are we happy?"
 22                ShortDescriptionLine2:""
 23                Description:"Harvard psychologist Dan Gilbert says our beliefs about what will make us happy are often wrong -- a premise he supports with intriguing research, and explains in his accessible and unexpectedly funny book, Stumbling on Happiness."
 24                Rating:"NR"
 25                StarRating:"80"
...

Unfortunately there's no syntax highlighting defined for Roku BrightScript. So I wrote my own.

The procedure goes roughly in two steps.

create: /usr/share/vim/vim72/syntax/brs.vim
add brs to: /usr/share/vim/vim72/filetype.vim

I've never created a vim syntax file before, so I decided to use the vb.vim syntax file as starter code. That served as a decent foundation to hack around and learn how the syntax file syntax works. While I was hacking the syntax file I kept a .brs program open in another instance of vim so that I could periodically reload the buffer (:e!) to see the effect of my changes. After all the hacking and experimentation, my brs.vim didn't very much resemble the vb.vim that I'd started with.

What I learned about the way the syntax file works:

There are a number of predefined categories of syntax which map to distinct highlight colors.
The syntax file defines keyword and pattern lists which are then related to any of the predefined syntax categories.
The patterns can be regexes such as . matches any character. (So use "\." if you want the . character to match.)
The syntax file has a lot of sophisticated capability which I didn't explore.

Adding the correct entry in filetype.vim was as simple as adding:

au BufNewFile,BufRead *.brs     setf brs

Here's my brs.vim if you'd like to take a look for yourself (last updated July 2011).

Doing It The jQuery Way · 509 days ago by Dylan Doxey

Here's (a slimmed version of) one of the more intersting bits of JavaScript code I've written recently.

    var refresh_ui = function() {

        $('#keyword_select').load('/keywords?action=list');
    }

    var save = function() {

        var item_id  = $('#item_select').val();
        var keywords = $('#keyword_select').val();

        return $.post( '/settings/keywords',
            {
                action   : 'save',
                item_id  : item_id,
                keywords : keywords,
            },
            refresh_ui
        );
    };

    $('#save_button').click(save);

This snippet illustrates several ways of doing things using jQuery that break from the old norm:

defining anonymous functions vs. named functions
describing data structures with inline JSON vs. building associative arrays
getting items with CSS selectors vs. getElementById (or something even more messy)
assigning event handlers by passing the function object directly
unspeakably more consice .load and .post functions.

I used to pride myself on writing fancy AJAX interfaces for each individual application. But golly, nothing I ever did approached the simplicity and elegance of just doing $('#item').load('/url/path');.

Don't reinvent the wheel. Stand on the shoulders of giants and move on to developing something else.

Dynamic Accessor Methods · 512 days ago by Dylan Doxey

So here I am writing this module which pulls a database record that is keyed by a topic keyword. And each accessor method was just a copy of the one preceding it, over and over. To make matters worse, each time there's a new topic added to the database am I to go an add yet another accessor method? Not.

Introducing the dynamic accessor method.

...

for my $topic (@topics) {

    my $method_name = sprintf '%s::get_%s', __PACKAGE__, $topic;

    my $method_rc = sub {
        my $self = shift;

        my $page_rh = $self->{db}->select_rh( {
            from     => 'content',
            where_rh => {
                topic => $topic,
            },
        } );

        $page_rh->{text} //= $MT_STR;

        return $page_rh->{text};
    };

    {
        no strict 'refs';
        *{$method_name} = $method_rc;
    }
}

...

Anyone curious about my database interface?

Changing Case in Vim · 517 days ago by Dylan Doxey

Every once in a while I'm preparing some text for something or other and I think, "I sure would like to change all of this text to lower case."

...

50 504 gateway_timeout
51 505 http_version_not_supported
52 506 variant_also_negotiates
53 507 insufficient_storage
54 509 bandwidth_limit_exceeded
55 510 not_extended
:'<,'>!tr A-Z a-z

This vim command is composed of three parts:

'<,'> -- this is the region of highlighted text.
! -- what follows is a shell command
tr A-Z a-z -- translate class [A-Z] to [a-z] (same as Perl tr/[A-Z]/[a-z]/).

I'm sure to be back for this snippet soon.

HTML Formatting Your Perl · 525 days ago by Dylan Doxey

Ever wonder how to make your Perl code presentable on a website? Use perltidy.

perltidy -html rand_sort.pl

Then cut & paste the resulting HTML into your blog and delight your readers.

#!/usr/bin/perl 

use strict;
use warnings;

my @numbers = sort { -1 + int rand 3 } 1 .. 1000;

for my $x (@numbers) {

    print "$x\n";
}

However, in my opinion the best option for a blog snippet is vim. While you've got your Perl (or whatever) file open in vim use the :TOhtml command. That will open another buffer with HTML in it. The resulting HTML page is:

 1 #!/usr/bin/perl 
 2 
 3 use strict;
 3 use warnings;
 4  
 5 my @numbers = sort { -1 + int rand 3 } 1 .. 1000; 
 6  
 7 for my $x (@numbers) { 
 8  
 9     print "$x\n"; 
10 } 
11

For more information about using perltidy see http://perltidy.sourceforge.net/perltidy.html#html_options. For the record, my example of perltidy HTML output is pretty lame. As linked from the perltidy home page (http://perltidy.sourceforge.net/) check out this example of perltidy HTML output: http://perltidy.sourceforge.net/Conf.pm.html.

For more details about using :TOhtml user command in vim see http://vimdoc.sourceforge.net/htmldoc/syntax.html#:TOhtml

Now I won't forget how to do that again!

Ejecting the CDROM · 587 days ago by Dylan Doxey

Well, the CDROM has been open for the last couple of hours.

I did a quick 'man eject' with hope that there's a way to close it without having to get out of my chair.

 eject -t

Ah, that's better.

SCP -- The World's Most Convenient File Transfer · 628 days ago by Dylan Doxey

Okay, world's most convenient might be a stretch if you don't do it very often. But if you can force yourself to remember this ...

scp -P 1234 favicon.ico yourself@example.com:/var/www/example.com/htdocs/

That's a relatively convenient way to push your brand new favicon.ico file to the server without all the drudgery of messing around with a GUI FTP client, or the rigmarole adding to your repository, committing, logging in on the server and updating.

And it goes the other way also. Need to fetch a report from your home directory?

scp -P 1234 yourself@example.com:/home/yourself/big_report.csv .

Now that's getting the job done without a lot of distraction!

Note: In your world you're likely change the port number 1234 to something else, or leave out the -P # option altogether.

Happy computing.