Steve Chervitz - CSHL Genome Informatics

Using Perl Modules

Steve A. Chervitz
Neomorphic, Inc. / The Bioperl Project
sac@neomorphic.com

Introduction

Perl modules provide a powerful way to extend the core features of the Perl language. They are easy to use and are not that hard to create. In this lecture, I'll first cover some basic information about modules and objects that will empower you to become fearless modules users.

Then, I'll introduce the Bioperl project. We'll see some of the Perl modules it has to offer for bioinformatics data processing and how to use them.

Lecture Notes

Using Perl Modules:

From references to objects
Why use Perl modules and objects?
What is a module?
Library modules versus object-oriented modules
Object-oriented concepts
Using library modules: Getopt
Using object-oriented modules: LWP
Where can I find useful modules?

Introduction to Bioperl:

Introducing The Bioperl Project
Documentation for Bioperl modules
Key objects: PrimarySeq, Seq, SeqIO
Survey of Tools
Restriction Enzyme object
How can I get involved in Bioperl?

Problem Sets

Why use Perl modules and objects?

Perl didn't always have modules and objects (they were introduced with Perl5). There are many "old-timers" that can get by without using modules and objects, designing scripts in so-called "procedural" style. So why should you consider using modules and objects?

Facilitates code reuse, prevents wheel re-invention
Increases software interoperability

Simplifies scripts. Makes it easier to write more complex applications

For example, here probably the world's shortest Fasta-to-EMBL sequence format converter:

    use Bio::SeqIO;

    $in  = Bio::SeqIO->newFh(-file => shift,  '-format' => 'Fasta');
    $out = Bio::SeqIO->newFh(-fh => \*STDOUT, '-format' => 'EMBL');

    print $out $_ while <$in>;

Allows you to "stand on the shoulders of giants"
- Publically available modules contain well-tested code written by domain experts used by many others in a variety of settings.
Tap into the world of software design patterns
Impress your friends ;-)

What is a module? What is a class? Object?

A Module is a collection of related subroutines and data designed to be used by other programs or modules. Modules are defined in separate files and use the package keyword to define a namespace for the data and subroutines in that module file.

A Module can be a simple collection of subroutines, not defining a class. Such modules are referred to as libraries.

A Class is a user-defined data type. Classes are defined in modules using special constructs that allow you to create variables of that type. The subroutines in these modules are referred to as methods.

An Object is a variable that belongs to a particular class. It contains its own, private data that can be operated on by the methods defined in the class. The object is called an instance of its class.

Example:

Module: Person.pm
Class: Person
Object: "Lincoln Stein"
Method: surname()

What's up with all the double colons (::)? The :: corresponds to a filesystem path separator. A Module called Shape::Rect corresponds to a file named Shape/Rect.pm relative to a module library directory.

From references to objects

A Perl object is just another data structure, nearly identical to a hash reference. You can use an object as a convenient way to store data.

Here's the familiar hash data structure:

         %hash = ( color => 'green',
                   size  => '17' );

         print "Color is $hash{color}\n";

Here's a hash reference data structure:

         $hash_ref = { color => 'green',
                       size  => '17' };

         # This is essentially equivalent to $hash = \%hash_ref;

         print "Color is $hash_ref->{color}\n";

Here's a object data structure:

         $object = Foo->new( -color=> 'green',
                             -size => '17' );

         print "Color is ", $object->get_color(), "\n";

So an object is nothing but a hash reference that knows how to call methods (new()) defined in a particular module (Foo.pm).

Object-oriented concepts

Separation of interface from implementation
An object (or module) has a public interface that allows users to call its subroutines without worrying about exactly how the operation is carried out. The interface consists of well-documented methods and is often referred to as a contract between the developer and the user.
Methods beginning with an underscore (e.g., _initialize())are considered private and should not be called by external users, although Perl cannot prevent this from happening.
Encapsulation
An object (or module) contains all the subroutines and data necessary to perform as a well-defined unit.

Using library modules: Getopt

Parsing of command-line arguments is such a common task that Perl comes with some standard modules for doing this: Getopt::Std and Getopt::Long. Recipe 15.1 in the Perl Cookbook shows how to use these. The basic idea is as follows:

        use Getopt::Long;

        die "Usage: $0 [-b] [-user name]\n" unless @ARGV;

        my( $binary, $username );

        GetOptions( "b"       => \$binary,
                    "user=s"  => \$username );

Using object-oriented modules: LWP

The LWP suite is an indispensible collection of modules for working with data from the internet. The modules are of the object-oriented type, so you create objects calling the new() method and then call various methods on the object you get back.

        #!/usr/bin/perl -w

        # Tell Perl we want to use the following modules:
        use LWP::UserAgent;
        use HTTP::Request;
        use HTTP::Response;
        use strict;          # always a good idea

        # Get data from @ARGV or die with a usage string.

        my $url = shift or die "Usage: $0 URL\n";

        # Create an instance of a LWP::UserAgent object

        my $ua = LWP::UserAgent->new();

        # Set some data on the UserAgent (the agent() method is a "setter")

        $ua->agent("Schmozilla/v9.14 Platinum");

        # Create an instance of a HTTP::Request object

        my $req = HTTP::Request->new( GET => $url );

        # Submit the HTTP::Request object to the UserAgent object
        # and get the response as another object (HTTP::Response)

        my $response = $ua->request( $req );

        # Then the $response object can be interrogated for its contents.
        # (See recipe 20.1 in the Perl Cookbook for how)

Other useful modules

File::Find - Recurse through all files in a given directory. This is a library-style module.
CGI.pm - For scripts that operate as CGI programs over the web. Works as either a library or an object-oriented module to suit programmers of either persuasion.
DBI.pm - Send SQL queries to a relational database and process results. Requires the appropriate DBD module (e.g., DBD::oracle, DBD::mysql).

Where can I find useful modules?

CPAN: Comprehensive Perl Archive Network. At search.cpan.org you can search by module name, author name, documentation, or distribution.
bioperl.org. Bioperl modules are also available via CPAN, but there is a lag after new releases.

Always check CPAN before embarking on a major Perl software development effort to see if there isn't already a module that does what you want to do. (Cross-fertilization between fields).

Recipes 12.7 and 12.17 in the Perl Cookbook have some useful tips for installing and using CPAN modules.

Introducing The Bioperl Project

Goals:

Encourage interoperability and prevent wheel reinvention among Perl-based computational biologists.
Collect computational methods routinely used in bioinformatics into a standard set of well-documented, reusable, and freely available Perl modules. Data-oriented.
Invite contributions from the bioinformatics community to improve and extend functionality within the Bioperl distribution.
Homepage: http://bio.perl.org

Documentation for Bioperl modules

There is a lot of code in the Bioperl arsenal. This can be intimidating at first. An excellent way to learn about a module is to see working examples.

There are several sources of sample bioperl code.

perldoc, e.g., execute at a shell prompt:
perldoc Bio::Seq

These are also available via the web at http://bio.perl.org/Core/Latest/modules.html. Perldoc can also be run on a module file directly (e.g., perldoc ~/perl/lib/Bio/Seq.pm), useful when the module isn't installed on your system.
The examples directory of the Bioperl distribution.
The t (test) directory of the Bioperl distribution.

For your convenience, a copy of the distribution directory is available locally: bioperl-0.6.2

Describing Object Relationships

Inheritance vs. composition. A class can inherit functionality (i.e., methods) from other classes. It can also contain objects beloging to other classes. Object relationships are most conveniently described using UML diagrams as shown below.

In this case, Child is a class that inherits functionality from Parent1, and Parent1 is a class that inherits functionality from GrandParent. Child is often called the subclass of Parent1, which is the superclass of Child. GrandParent is a superclass of both Parent1 and Child.

Child also inherits functionality from Parent2, so it has two superclasses (this is a case of multiple inheritance) which is permitted in Perl.

The diagram also indicates that Child can contain a reference to an object of type ContainedByChild.

For more information about UML, an established standard, see http://directory.google.com/Top/Computers/Software/Object_Oriented/Methodologies/UML/.

Key Bioperl objects

Bio::PrimarySeq

     use Bio::PrimarySeq;

     # Create a new PrimarySeq object

     $seqobj = Bio::PrimarySeq->new(-seq     => 'ACTGTGGCGTCAACTG', 
                                    -moltype => 'dna', 
                                    -id      => 'Primer-22301');

     # Get a substring from it

     $substring = $seqobj->subseq( 2, 4 );

Bio::Seq

Bio::SeqIO

    use Bio::SeqIO;

    # Note that we don't have to use Bio::Seq in order to work with
    # Bio::Seq objects.

    # To create a Bio::Seq object you must first create a SeqIO object

    $seqio  = Bio::SeqIO->new ( '-format' => 'Fasta' , 
                                '-file'   => 'myfile.fasta');

    # Get the first sequence in the file.
    $seqobj = $seqio->next_seq();

    # Get the actual sequence as a string
    $seq      = $seqobj->seq(); 

    # Bio::Seq also inherits the same subseq() method from PrimarySeq
    $seqstr   = $seqobj->subseq(10,50);    

    # Iterate through the remaining sequences in the file

    while ( my $seq = $seqio->next_seq() ) {
        # Get the sequence as a string for some analysis
        my ($id, $str) = $seq->display_id, $seq->seq;
    }

Bio::SeqFeatureI

Bio::Seq objects also may have features attached to them. Bioperl features implement the methods defined by Bio::SeqFeatureI.

    # Get top level features

    @features = $seqobj->top_SeqFeatures(); 

    # Descend into sub features (features may themselves have features)

    @features = $seqobj->all_SeqFeatures(); 

    # Get an Annotation object
    $ann = $seqobj->annotation();

Annotation objects are a recent addition to Bioperl and provide:

a brief, one line description
free text comments
links to other biological objects
references to literature

Survey of Bioperl Tools

Bio::Tools::AlignFactory.pm

Bio::Tools::Blast.pm

Bio::Tools::SeqStats.pm

Bio::Tools::SeqPattern.pm

Bio::Tools::SigCleave.pm

Bio::Tools::RestrictionEnzyme.pm

Bio::Tools::HMMER::Results.pm

Bio::Tools::WWW.pm

Bio::Tools::CodonTable.pm

Bio::Tools::IUPAC.pm

Bioperl Restriction Enzyme object The Bio::Tools::RestrictionEnzyme module represents a restriction endonuclease. It can conceptually "cut" a Bio::Seq into fragments based on specificity of an actual enzyme. It allows for the construction of objects representing about 150 different REs. There's also a mechanism for defining the recognition sequence when creating the object, to cover new enzymes. use Bio::Seq; use Bio::Tools::RestrictionEnzyme; # Get a sequence string from somewhere. $sequence = get_sequence(); # Create a Bio::Seq object using the string $seq = new Bio::Seq( -ID => 'test_seq', -SEQ => $sequence); $re = new Bio::Tools::RestrictionEnzyme( -name => 'EcoRI' ); ## Cut the sequence with the restriction enzyme object. @fragments = $re->cut_seq( $seq );

How can I get involved in Bioperl?

As a user: Report bugs, critique functionality. Join the Bioperl discussion mailing list.
As developer: Experiment using and writing modules, learn more object-oriented design principles (Recent book on OO Perl by Damian Conway), learn cvs. Join the Bioperl guts mailing list.