checkinstall

Checkinstall is a nice tool to create simple .deb-packages that you can use in your local network (e.g. if you have to install the same piece of software on multiple computers running Debian). It lets you compile and install software from the sources like before, but with the difference that you end up with a simple Debian package which also means that you can easily uninstall the software you just compiled by running dpkg -r!

Install Checkinstall It is as easy as 1-2-3:
apt-get install checkinstall
If your system tells you that it does not know a package called checkinstall then add the following line to /etc/apt/sources.list:
deb http://www.backports.org/debian/ woody checkinstall
and run
apt-get update
Then try again to install checkinstall.

Valgrind

Valgrind Valgrind is a memory mismanagement detector. It shows you memory leaks, deallocation errors, etc. Actually, Valgrind is a wrapper around a collection of tools that do many other things (e.g., cache profiling); however, here we focus on the default tool, memcheck. Memcheck can detect:

• Use of uninitialised memory
• Reading/writing memory after it has been free'd
• Reading/writing off the end of malloc'd blocks
• Reading/writing inappropriate areas on the stack
• Memory leaks -- where pointers to malloc'd blocks are lost forever
• Mismatched use of malloc/new/new [] vs free/delete/delete []
• Overlapping src and dst pointers in memcpy() and related functions
• Some misuses of the POSIX pthreads API

To use this on our example program, test.c, try

gcc -o test -g test.c

This creates an executable named test. To check for memory leaks during the execution of test, try

valgrind --tool=memcheck --leak-check=yes --show-reachable=yes --num-callers=20 --track-fds=yes ./test

This outputs a report to the terminal like

==9704== Memcheck, a memory error detector for x86-linux.
==9704== Copyright (C) 2002-2004, and GNU GPL'd, by Julian Seward et al.
==9704== Using valgrind-2.2.0, a program supervision framework for x86-linux.
==9704== Copyright (C) 2000-2004, and GNU GPL'd, by Julian Seward et al.
==9704== For more details, rerun with: -v
==9704== 
==9704== 
==9704== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 11 from 1)
==9704== malloc/free: in use at exit: 35 bytes in 2 blocks.
==9704== malloc/free: 3 allocs, 1 frees, 47 bytes allocated.
==9704== For counts of detected errors, rerun with: -v
==9704== searching for pointers to 2 not-freed blocks.
==9704== checked 1420940 bytes.
==9704== 
==9704== 16 bytes in 1 blocks are definitely lost in loss record 1 of 2
==9704==    at 0x1B903D38: malloc (vg_replace_malloc.c:131)
==9704==    by 0x80483BF: main (test.c:15)
==9704== 
==9704== 
==9704== 19 bytes in 1 blocks are definitely lost in loss record 2 of 2
==9704==    at 0x1B903D38: malloc (vg_replace_malloc.c:131)
==9704==    by 0x8048391: main (test.c:8)
==9704== 
==9704== LEAK SUMMARY:
==9704==    definitely lost: 35 bytes in 2 blocks.
==9704==    possibly lost:   0 bytes in 0 blocks.
==9704==    still reachable: 0 bytes in 0 blocks.
==9704==         suppressed: 0 bytes in 0 blocks.

Let's look at the code to see what happened. Allocation #1 (19 byte leak) is lost because p is pointed elsewhere before the memory from Allocation #1 is free'd. To help us track it down, Valgrind gives us a stack trace showing where the bytes were allocated. In the 19 byte leak entry, the bytes were allocate in test.c, line 8. Allocation #2 (12 byte leak) doesn't show up in the list because it is free'd. Allocation #3 shows up in the list even though there is still a reference to it (p) at program termination. This is still a memory leak! Again, Valgrind tells us where to look for the allocation (test.c line 15).

World repo - the repository of the world

 This post is selected python conference held at bangalore ,2010.

Authors V.Srikrishnan, V. Shunmugachamy
Level Intermediate
Topic Software development tools
Tags urllib, regular expressions,

Summary


WorldRepo is a python based search engine, which gives you the exact active URL for download.It maintains a repository containing the active links for the user to download. unlike conventional linux distros which provide a repository of software only for their OS and environment, WorldRepo contains a huge database of all available software/Movies/study materials in one place.

It is one gigantic repository for all downloadable entities. Also the availability of the downloadable links is checked constantly and updated. Developers would love this tool for it enables them to find any software through one interface.
Outline
Introduction to WorldRepo(A Python based search engine)
Features of WorldRepo(Basics of Crawling and Database Programming)
Customization of WorldRepo(Usage of crontab in Linux)
Downloading a File(Usage of urllib2 )
Automating the Download and intimating the User once the download is completed(Usage of crontab and smtplib)
Enhancing Worldrepo( Enhancing a Search Engine for Developer Community )
Some Hands on Session on Crawling,Crontab,Database Programming,urllib2
Notes
Profile of the authors

Both V.Srikrishnan and V.Shunmugachamy are working at CDAC (Centre for development of advanced computing, chennai) in the NRCFOSS project (National Resource centre for free and opensource software) funded by the Department of Information technology, Government of India.

They are involved in the development of various open source tools. Mr V.Srikrishnan is presently working in building an open source cloud computing environment, a first of its kind, at CDAC chennai.

Mr. V. Shunmugachamy is involved in the design of common desktop development which targets interoperability between KDE and GNOME desktops.

Their interest in python stems from their liking for opensource tools and their belief in the strength of the opensource community available for python.

WorldRepo had been developed with the spare time available after their research endeavors at work. It has been developed both as a useful tool for the community and also to demonstrate the power of Python and its in-built modules like RE, URLLIB2, SYS, etc.
Filesfile size uploaded comment
world-repo--final.pdf 293.5 KB september 24 World Repo - Presentation Slide
world-repo-final.pdf 405.3 KB september 10 World Repo - Presentation Slide

DJANGO FRAMEWORK-FORM PROCESSING LIGHTLY

Web-poll application and will focus on simple form processing and cutting down our code.

Write a simple form

Let’s update our poll detail template (“polls/detail.html”) from the last tutorial, so that the template contains an HTML

element:

{{ poll.question }}
{% if error_message %}{{ error_message }}{% endif %}

 action="/polls/{{ poll.id }}/vote/" method="post">
{% csrf_token %}
{% for choice in poll.choice_set.all %}
     type="radio" name="choice" id="choice{{ forloop.counter }}" value="{{ choice.id }}" />
     />
{% endfor %}
 type="submit" value="Vote" />

A quick rundown:

• The above template displays a radio button for each poll choice. The value of each radio button is the associated poll choice's ID. The name of each radio button is "choice". That means, when somebody selects one of the radio buttons and submits the form, it'll send the POST data choice=3. This is HTML Forms 101.
• We set the form's action to /polls/{{ poll.id }}/vote/, and we set method="post". Using method="post" (as opposed to method="get") is very important, because the act of submitting this form will alter data server-side. Whenever you create a form that alters data server-side, use method="post". This tip isn't specific to Django; it's just good Web development practice.
• forloop.counter indicates how many times the for tag has gone through its loop
• Since we're creating a POST form (which can have the effect of modifying data), we need to worry about Cross Site Request Forgeries. Thankfully, you don't have to worry too hard, because Django comes with a very easy-to-use system for protecting against it. In short, all POST forms that are targeted at internal URLs should use the {% csrf_token %} template tag.

The {% csrf_token %} tag requires information from the request object, which is not normally accessible from within the template context. To fix this, a small adjustment needs to be made to the detail view, so that it looks like the following:

from django.template import RequestContext
# ...
def detail(request, poll_id):
    p = get_object_or_404(Poll, pk=poll_id)
    return render_to_response('polls/detail.html', {'poll': p},
                               context_instance=RequestContext(request))

The details of how this works are explained in the documentation for RequestContext.
Now, let's create a Django view that handles the submitted data and does something with it. Remember, in Tutorial 3, we created a URLconf for the polls application that includes this line:

(r'^(?P\d+)/vote/$', 'vote'),

We also created a dummy implementation of the vote() function. Let's create a real version. Add the following to mysite/polls/views.py:

from django.shortcuts import get_object_or_404, render_to_response
from django.http import HttpResponseRedirect, HttpResponse
from django.core.urlresolvers import reverse
from django.template import RequestContext
from mysite.polls.models import Choice, Poll
# ...
def vote(request, poll_id):
    p = get_object_or_404(Poll, pk=poll_id)
    try:
        selected_choice = p.choice_set.get(pk=request.POST['choice'])
    except (KeyError, Choice.DoesNotExist):
        # Redisplay the poll voting form.
        return render_to_response('polls/detail.html', {
            'poll': p,
            'error_message': "You didn't select a choice.",
        }, context_instance=RequestContext(request))
    else:
        selected_choice.votes += 1
        selected_choice.save()
        # Always return an HttpResponseRedirect after successfully dealing
        # with POST data. This prevents data from being posted twice if a
        # user hits the Back button.
        return HttpResponseRedirect(reverse('mysite.polls.views.results', args=(p.id,)))

This code includes a few things we haven't covered yet in this tutorial:

• 
  
  request.POST is a dictionary-like object that lets you access submitted data by key name. In this case, request.POST['choice'] returns the ID of the selected choice, as a string. request.POST values are always strings.
  Note that Django also provides request.GET for accessing GET data in the same way -- but we're explicitly using request.POST in our code, to ensure that data is only altered via a POST call.
  
• 
  
  request.POST['choice'] will raise KeyError if choice wasn't provided in POST data. The above code checks for KeyError and redisplays the poll form with an error message if choice isn't given.
• 
  
  After incrementing the choice count, the code returns an HttpResponseRedirect rather than a normal HttpResponse. HttpResponseRedirect takes a single argument: the URL to which the user will be redirected (see the following point for how we construct the URL in this case).
  As the Python comment above points out, you should always return an HttpResponseRedirect after successfully dealing with POST data. This tip isn't specific to Django; it's just good Web development practice.
  
• 
  
  We are using the reverse() function in the HttpResponseRedirect constructor in this example. This function helps avoid having to hardcode a URL in the view function. It is given the name of the view that we want to pass control to and the variable portion of the URL pattern that points to that view. In this case, using the URLconf we set up in Tutorial 3, this reverse() call will return a string like

  '/polls/3/results/'

  ... where the 3 is the value of p.id. This redirected URL will then call the 'results' view to display the final page. Note that you need to use the full name of the view here (including the prefix).
  

As mentioned in Tutorial 3, request is a HttpRequest object. For more on HttpRequest objects, see the request and response documentation.
After somebody votes in a poll, the vote() view redirects to the results page for the poll. Let's write that view:

def results(request, poll_id):
    p = get_object_or_404(Poll, pk=poll_id)
    return render_to_response('polls/results.html', {'poll': p})

This is almost exactly the same as the detail() view from Tutorial 3. The only difference is the template name. We'll fix this redundancy later.
Now, create a results.html template:

{{ poll.question }}



{% for choice in poll.choice_set.all %}     

{{ choice.choice }} -- {{ choice.votes }} vote{{ choice.votes|pluralize }} {% endfor %} 


 href="/polls/{{ poll.id }}/">Vote again?

Now, go to /polls/1/ in your browser and vote in the poll. You should see a results page that gets updated each time you vote. If you submit the form without having chosen a choice, you should see the error message.

Use generic views: Less code is better

The detail() (from Tutorial 3) and results() views are stupidly simple -- and, as mentioned above, redundant. The index() view (also from Tutorial 3), which displays a list of polls, is similar.
These views represent a common case of basic Web development: getting data from the database according to a parameter passed in the URL, loading a template and returning the rendered template. Because this is so common, Django provides a shortcut, called the "generic views" system.
Generic views abstract common patterns to the point where you don't even need to write Python code to write an app.
Let's convert our poll app to use the generic views system, so we can delete a bunch of our own code. We'll just have to take a few steps to make the conversion. We will:

1. Convert the URLconf.
2. Rename a few templates.
3. Delete some of the old, unneeded views.
4. Fix up URL handling for the new views.

Read on for details.

Why the code-shuffle?

Generally, when writing a Django app, you'll evaluate whether generic views are a good fit for your problem, and you'll use them from the beginning, rather than refactoring your code halfway through. But this tutorial intentionally has focused on writing the views "the hard way" until now, to focus on core concepts.

You should know basic math before you start using a calculator.

First, open the polls/urls.py URLconf. It looks like this, according to the tutorial so far:

from django.conf.urls.defaults import *

urlpatterns = patterns('mysite.polls.views',
    (r'^$', 'index'),
    (r'^(?P\d+)/$', 'detail'),
    (r'^(?P\d+)/results/$', 'results'),
    (r'^(?P\d+)/vote/$', 'vote'),
)

Change it like so:

from django.conf.urls.defaults import *
from mysite.polls.models import Poll

info_dict = {
    'queryset': Poll.objects.all(),
}

urlpatterns = patterns('',
    (r'^$', 'django.views.generic.list_detail.object_list', info_dict),
    (r'^(?P\d+)/$', 'django.views.generic.list_detail.object_detail', info_dict),
    url(r'^(?P\d+)/results/$', 'django.views.generic.list_detail.object_detail', dict(info_dict, template_name='polls/results.html'), 'poll_results'),
    (r'^(?P\d+)/vote/$', 'mysite.polls.views.vote'),
)

We're using two generic views here: object_list() and object_detail(). Respectively, those two views abstract the concepts of "display a list of objects" and "display a detail page for a particular type of object."

• Each generic view needs to know what data it will be acting upon. This data is provided in a dictionary. The queryset key in this dictionary points to the list of objects to be manipulated by the generic view.
• The object_detail() generic view expects the ID value captured from the URL to be called "object_id", so we've changed poll_id to object_id for the generic views.
• We've added a name, poll_results, to the results view so that we have a way to refer to its URL later on (see the documentation about naming URL patterns for information). We're also using the url() function from django.conf.urls.defaults here. It's a good habit to use url() when you are providing a pattern name like this.

By default, the object_detail() generic view uses a template called name>/ name>_detail.html. In our case, it'll use the template "polls/poll_detail.html". Thus, rename your polls/detail.html template to polls/poll_detail.html, and change the render_to_response() line in vote().
Similarly, the object_list() generic view uses a template called name>/ name>_list.html. Thus, rename polls/index.html to polls/poll_list.html.
Because we have more than one entry in the URLconf that uses object_detail() for the polls app, we manually specify a template name for the results view: template_name='polls/results.html'. Otherwise, both views would use the same template. Note that we use dict() to return an altered dictionary in place.

Note

django.db.models.QuerySet.all() is lazy
It might look a little frightening to see Poll.objects.all() being used in a detail view which only needs one Poll object, but don't worry; Poll.objects.all() is actually a special object called a QuerySet, which is "lazy" and doesn't hit your database until it absolutely has to. By the time the database query happens, the object_detail() generic view will have narrowed its scope down to a single object, so the eventual query will only select one row from the database.

If you'd like to know more about how that works, The Django database API documentation explains the lazy nature of QuerySet objects.

In previous parts of the tutorial, the templates have been provided with a context that contains the poll and latest_poll_list context variables. However, the generic views provide the variables object and object_list as context. Therefore, you need to change your templates to match the new context variables. Go through your templates, and modify any reference to latest_poll_list to object_list, and change any reference to poll to object.
You can now delete the index(), detail() and results() views from polls/views.py. We don't need them anymore -- they have been replaced by generic views.
The vote() view is still required. However, it must be modified to match the new context variables. In the render_to_response() call, rename the poll context variable to object.
The last thing to do is fix the URL handling to account for the use of generic views. In the vote view above, we used the reverse() function to avoid hard-coding our URLs. Now that we've switched to a generic view, we'll need to change the reverse() call to point back to our new generic view. We can't simply use the view function anymore -- generic views can be (and are) used multiple times -- but we can use the name we've given:

return HttpResponseRedirect(reverse('poll_results', args=(p.id,)))

Run the server, and use your new polling app based on generic views.

DJANGO FRAMEWORK-CREATING VIEWS

 

Philosophy

A view is a “type” of Web page in your Django application that generally serves a specific function and has a specific template. For example, in a weblog application, you might have the following views:

• Blog homepage – displays the latest few entries.
• Entry “detail” page – permalink page for a single entry.
• Year-based archive page – displays all months with entries in the given year.
• Month-based archive page – displays all days with entries in the given month.
• Day-based archive page – displays all entries in the given day.
• Comment action – handles posting comments to a given entry.

In our poll application, we’ll have the following four views:

• Poll “archive” page – displays the latest few polls.
• Poll “detail” page – displays a poll question, with no results but with a form to vote.
• Poll “results” page – displays results for a particular poll.
• Vote action – handles voting for a particular choice in a particular poll.

In Django, each view is represented by a simple Python function.

Design your URLs

The first step of writing views is to design your URL structure. You do this by creating a Python module, called a URLconf. URLconfs are how Django associates a given URL with given Python code.
When a user requests a Django-powered page, the system looks at the ROOT_URLCONF setting, which contains a string in Python dotted syntax. Django loads that module and looks for a module-level variable called urlpatterns, which is a sequence of tuples in the following format:

(regular expression, Python callback function [, optional dictionary])

Django starts at the first regular expression and makes its way down the list, comparing the requested URL against each regular expression until it finds one that matches.
When it finds a match, Django calls the Python callback function, with an HttpRequest object as the first argument, any "captured" values from the regular expression as keyword arguments, and, optionally, arbitrary keyword arguments from the dictionary (an optional third item in the tuple).
For more on HttpRequest objects, see the Request and response objects. For more details on URLconfs, see the URL dispatcher.
When you ran django-admin.py startproject mysite at the beginning of Tutorial 1, it created a default URLconf in mysite/urls.py. It also automatically set your ROOT_URLCONF setting (in settings.py) to point at that file:

ROOT_URLCONF = 'mysite.urls'

Time for an example. Edit mysite/urls.py so it looks like this:

from django.conf.urls.defaults import *

from django.contrib import admin
admin.autodiscover()

urlpatterns = patterns('',
    (r'^polls/$', 'mysite.polls.views.index'),
    (r'^polls/(?P\d+)/$', 'mysite.polls.views.detail'),
    (r'^polls/(?P\d+)/results/$', 'mysite.polls.views.results'),
    (r'^polls/(?P\d+)/vote/$', 'mysite.polls.views.vote'),
    (r'^admin/', include(admin.site.urls)),
)

This is worth a review. When somebody requests a page from your Web site -- say, "/polls/23/", Django will load this Python module, because it's pointed to by the ROOT_URLCONF setting. It finds the variable named urlpatterns and traverses the regular expressions in order. When it finds a regular expression that matches -- r'^polls/(?P\d+)/$' -- it loads the function detail() from mysite/polls/views.py. Finally, it calls that detail() function like so:

detail(request=, poll_id='23')

The poll_id='23' part comes from (?P\d+). Using parentheses around a pattern "captures" the text matched by that pattern and sends it as an argument to the view function; the ?P defines the name that will be used to identify the matched pattern; and \d+ is a regular expression to match a sequence of digits (i.e., a number).
Because the URL patterns are regular expressions, there really is no limit on what you can do with them. And there's no need to add URL cruft such as .php -- unless you have a sick sense of humor, in which case you can do something like this:

(r'^polls/latest\.php$', 'mysite.polls.views.index'),

But, don't do that. It's silly.
Note that these regular expressions do not search GET and POST parameters, or the domain name. For example, in a request to http://www.example.com/myapp/, the URLconf will look for myapp/. In a request to http://www.example.com/myapp/?page=3, the URLconf will look for myapp/.
If you need help with regular expressions, see Wikipedia's entry and the Python documentation. Also, the O'Reilly book "Mastering Regular Expressions" by Jeffrey Friedl is fantastic.
Finally, a performance note: these regular expressions are compiled the first time the URLconf module is loaded. They're super fast.

Write your first view

Well, we haven't created any views yet -- we just have the URLconf. But let's make sure Django is following the URLconf properly.
Fire up the Django development Web server:

python manage.py runserver

Now go to "http://localhost:8000/polls/" on your domain in your Web browser. You should get a pleasantly-colored error page with the following message:

ViewDoesNotExist at /polls/

Tried index in module mysite.polls.views. Error was: 'module'
object has no attribute 'index'

This error happened because you haven't written a function index() in the module mysite/polls/views.py.
Try "/polls/23/", "/polls/23/results/" and "/polls/23/vote/". The error messages tell you which view Django tried (and failed to find, because you haven't written any views yet).
Time to write the first view. Open the file mysite/polls/views.py and put the following Python code in it:

from django.http import HttpResponse

def index(request):
    return HttpResponse("Hello, world. You're at the poll index.")

This is the simplest view possible. Go to "/polls/" in your browser, and you should see your text.
Now lets add a few more views. These views are slightly different, because they take an argument (which, remember, is passed in from whatever was captured by the regular expression in the URLconf):

def detail(request, poll_id):
    return HttpResponse("You're looking at poll %s." % poll_id)

def results(request, poll_id):
    return HttpResponse("You're looking at the results of poll %s." % poll_id)

def vote(request, poll_id):
    return HttpResponse("You're voting on poll %s." % poll_id)

Take a look in your browser, at "/polls/34/". It'll run the detail() method and display whatever ID you provide in the URL. Try "/polls/34/results/" and "/polls/34/vote/" too -- these will display the placeholder results and voting pages.

Write views that actually do something

Each view is responsible for doing one of two things: Returning an HttpResponse object containing the content for the requested page, or raising an exception such as Http404. The rest is up to you.
Your view can read records from a database, or not. It can use a template system such as Django's -- or a third-party Python template system -- or not. It can generate a PDF file, output XML, create a ZIP file on the fly, anything you want, using whatever Python libraries you want.
All Django wants is that HttpResponse. Or an exception.
Because it's convenient, let's use Django's own database API, which we covered in Tutorial 1. Here's one stab at the index() view, which displays the latest 5 poll questions in the system, separated by commas, according to publication date:

from mysite.polls.models import Poll
from django.http import HttpResponse

def index(request):
    latest_poll_list = Poll.objects.all().order_by('-pub_date')[:5]
    output = ', '.join([p.question for p in latest_poll_list])
    return HttpResponse(output)

There's a problem here, though: The page's design is hard-coded in the view. If you want to change the way the page looks, you'll have to edit this Python code. So let's use Django's template system to separate the design from Python:

from django.template import Context, loader
from mysite.polls.models import Poll
from django.http import HttpResponse

def index(request):
    latest_poll_list = Poll.objects.all().order_by('-pub_date')[:5]
    t = loader.get_template('polls/index.html')
    c = Context({
        'latest_poll_list': latest_poll_list,
    })
    return HttpResponse(t.render(c))

That code loads the template called "polls/index.html" and passes it a context. The context is a dictionary mapping template variable names to Python objects.
Reload the page. Now you'll see an error:

TemplateDoesNotExist at /polls/
polls/index.html

Ah. There's no template yet. First, create a directory, somewhere on your filesystem, whose contents Django can access. (Django runs as whatever user your server runs.) Don't put them under your document root, though. You probably shouldn't make them public, just for security's sake. Then edit TEMPLATE_DIRS in your settings.py to tell Django where it can find templates -- just as you did in the "Customize the admin look and feel" section of Tutorial 2.
When you've done that, create a directory polls in your template directory. Within that, create a file called index.html. Note that our loader.get_template('polls/index.html') code from above maps to "[template_directory]/polls/index.html" on the filesystem.
Put the following code in that template:

{% if latest_poll_list %}
    
{% for poll in latest_poll_list %}         

 href="/polls/{{ poll.id }}/">{{ poll.question }}     {% endfor %}     
{% else %}
    No polls are available.
{% endif %}

Load the page in your Web browser, and you should see a bulleted-list containing the "What's up" poll from Tutorial 1. The link points to the poll's detail page.

A shortcut: render_to_response()

It's a very common idiom to load a template, fill a context and return an HttpResponse object with the result of the rendered template. Django provides a shortcut. Here's the full index() view, rewritten:

from django.shortcuts import render_to_response
from mysite.polls.models import Poll

def index(request):
    latest_poll_list = Poll.objects.all().order_by('-pub_date')[:5]
    return render_to_response('polls/index.html', {'latest_poll_list': latest_poll_list})

Note that once we've done this in all these views, we no longer need to import loader, Context and HttpResponse.
The render_to_response() function takes a template name as its first argument and a dictionary as its optional second argument. It returns an HttpResponse object of the given template rendered with the given context.

Raising 404

Now, let's tackle the poll detail view -- the page that displays the question for a given poll. Here's the view:

from django.http import Http404
# ...
def detail(request, poll_id):
    try:
        p = Poll.objects.get(pk=poll_id)
    except Poll.DoesNotExist:
        raise Http404
    return render_to_response('polls/detail.html', {'poll': p})

The new concept here: The view raises the Http404 exception if a poll with the requested ID doesn't exist.
We'll discuss what you could put in that polls/detail.html template a bit later, but if you'd like to quickly get the above example working, just:

{{ poll }}

will get you started for now.

A shortcut: get_object_or_404()

It's a very common idiom to use get() and raise Http404 if the object doesn't exist. Django provides a shortcut. Here's the detail() view, rewritten:

from django.shortcuts import render_to_response, get_object_or_404
# ...
def detail(request, poll_id):
    p = get_object_or_404(Poll, pk=poll_id)
    return render_to_response('polls/detail.html', {'poll': p})

The get_object_or_404() function takes a Django model as its first argument and an arbitrary number of keyword arguments, which it passes to the module's get() function. It raises Http404 if the object doesn't exist.

Philosophy

Why do we use a helper function get_object_or_404() instead of automatically catching the ObjectDoesNotExist exceptions at a higher level, or having the model API raise Http404 instead of ObjectDoesNotExist?

Because that would couple the model layer to the view layer. One of the foremost design goals of Django is to maintain loose coupling.

There's also a get_list_or_404() function, which works just as get_object_or_404() -- except using filter() instead of get(). It raises Http404 if the list is empty.

Write a 404 (page not found) view

When you raise Http404 from within a view, Django will load a special view devoted to handling 404 errors. It finds it by looking for the variable handler404, which is a string in Python dotted syntax -- the same format the normal URLconf callbacks use. A 404 view itself has nothing special: It's just a normal view.
You normally won't have to bother with writing 404 views. By default, URLconfs have the following line up top:

from django.conf.urls.defaults import *

That takes care of setting handler404 in the current module. As you can see in django/conf/urls/defaults.py, handler404 is set to django.views.defaults.page_not_found() by default.
Four more things to note about 404 views:

• If DEBUG is set to True (in your settings module) then your 404 view will never be used (and thus the 404.html template will never be rendered) because the traceback will be displayed instead.
• The 404 view is also called if Django doesn't find a match after checking every regular expression in the URLconf.
• If you don't define your own 404 view -- and simply use the default, which is recommended -- you still have one obligation: To create a 404.html template in the root of your template directory. The default 404 view will use that template for all 404 errors.
• If DEBUG is set to False (in your settings module) and if you didn't create a 404.html file, an Http500 is raised instead. So remember to create a 404.html.

Write a 500 (server error) view

Similarly, URLconfs may define a handler500, which points to a view to call in case of server errors. Server errors happen when you have runtime errors in view code.

Use the template system

Back to the detail() view for our poll application. Given the context variable poll, here's what the "polls/detail.html" template might look like:

{{ poll.question }}
{% for choice in poll.choice_set.all %}     

{{ choice.choice }} {% endfor %} 

The template system uses dot-lookup syntax to access variable attributes. In the example of {{ poll.question }}, first Django does a dictionary lookup on the object poll. Failing that, it tries attribute lookup -- which works, in this case. If attribute lookup had failed, it would've tried calling the method question() on the poll object.
Method-calling happens in the {% for %} loop: poll.choice_set.all is interpreted as the Python code poll.choice_set.all(), which returns an iterable of Choice objects and is suitable for use in the {% for %} tag.

Simplifying the URLconfs

Take some time to play around with the views and template system. As you edit the URLconf, you may notice there's a fair bit of redundancy in it:

urlpatterns = patterns('',
    (r'^polls/$', 'mysite.polls.views.index'),
    (r'^polls/(?P\d+)/$', 'mysite.polls.views.detail'),
    (r'^polls/(?P\d+)/results/$', 'mysite.polls.views.results'),
    (r'^polls/(?P\d+)/vote/$', 'mysite.polls.views.vote'),
)

Namely, mysite.polls.views is in every callback.
Because this is a common case, the URLconf framework provides a shortcut for common prefixes. You can factor out the common prefixes and add them as the first argument to patterns(), like so:

urlpatterns = patterns('mysite.polls.views',
    (r'^polls/$', 'index'),
    (r'^polls/(?P\d+)/$', 'detail'),
    (r'^polls/(?P\d+)/results/$', 'results'),
    (r'^polls/(?P\d+)/vote/$', 'vote'),
)

This is functionally identical to the previous formatting. It's just a bit tidier.

Decoupling the URLconfs

While we're at it, we should take the time to decouple our poll-app URLs from our Django project configuration. Django apps are meant to be pluggable -- that is, each particular app should be transferable to another Django installation with minimal fuss.
Our poll app is pretty decoupled at this point, thanks to the strict directory structure that python manage.py startapp created, but one part of it is coupled to the Django settings: The URLconf.
We've been editing the URLs in mysite/urls.py, but the URL design of an app is specific to the app, not to the Django installation -- so let's move the URLs within the app directory.
Copy the file mysite/urls.py to mysite/polls/urls.py. Then, change mysite/urls.py to remove the poll-specific URLs and insert an include():

# ...
urlpatterns = patterns('',
    (r'^polls/', include('mysite.polls.urls')),
    # ...

include(), simply, references another URLconf. Note that the regular expression doesn't have a $ (end-of-string match character) but has the trailing slash. Whenever Django encounters include(), it chops off whatever part of the URL matched up to that point and sends the remaining string to the included URLconf for further processing.
Here's what happens if a user goes to "/polls/34/" in this system:

• Django will find the match at '^polls/'
• Then, Django will strip off the matching text ("polls/") and send the remaining text -- "34/" -- to the 'mysite.polls.urls' URLconf for further processing.

Now that we've decoupled that, we need to decouple the 'mysite.polls.urls' URLconf by removing the leading "polls/" from each line, and removing the lines registering the admin site:

urlpatterns = patterns('mysite.polls.views',
    (r'^$', 'index'),
    (r'^(?P\d+)/$', 'detail'),
    (r'^(?P\d+)/results/$', 'results'),
    (r'^(?P\d+)/vote/$', 'vote'),
)

The idea behind include() and URLconf decoupling is to make it easy to plug-and-play URLs. Now that polls are in their own URLconf, they can be placed under "/polls/", or under "/fun_polls/", or under "/content/polls/", or any other URL root, and the app will still work.

SSH login without password

Your aim

You want to use Linux and OpenSSH to automize your tasks. Therefore you need an
 automatic login from host A / user a to Host B / user b. You don't want to enter any
 passwords, because you want to call ssh from a within a shell script.

How to do it

First log in on A as user a and generate a pair of authentication keys. Do not enter a passphrase:

a@A:~> ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/a/.ssh/id_rsa): 
Created directory '/home/a/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/a/.ssh/id_rsa.
Your public key has been saved in /home/a/.ssh/id_rsa.pub.
The key fingerprint is:
3e:4f:05:79:3a:9f:96:7c:3b:ad:e9:58:37:bc:37:e4 a@A

Now use ssh to create a directory ~/.ssh as
user b on B. (The directory may already exist, which is fine):

a@A:~> ssh b@B mkdir -p .ssh
b@B's password: 
Finally append a's new public key to b@B:.ssh/authorized_keys and
enter b's password one last time:

a@A:~> cat .ssh/id_rsa.pub | ssh b@B 'cat >> .ssh/authorized_keys'
b@B's password: 
From now on you can log into B as b from A as a without password:

a@A:~> ssh b@B hostname
B
A

note from one of our readers: Depending on your version of SSH
you might also have to do the following changes:

Put the public key in .ssh/authorized_keys2  
Change the permissions of .ssh to 700  
Change the permissions of .ssh/authorized_keys2 to 640  

NoSQL(Not only SQL) A Relational Database Management System

NoSQL is a fast, portable, relational database management system without arbitrary limits, (other than memory and processor speed) that runs under, and interacts with, the UNIX¹ Operating System. It uses the "Operator-Stream Paradigm" described in "Unix Review", March, 1991, page 24, entitled "A 4GL Language". There are a number of "operators" that each perform a unique function on the data. The "stream" is supplied by the UNIX Input/Output redirection mechanism. Therefore each operator processes some data and then passes it along to the next operator via the UNIX pipe function. This is very efficient as UNIX pipes are implemented in memory. NoSQL is compliant with the "Relational Model".

What is NoSQL

NoSQL, which I personally like to pronounce as noseequel², is a derivative of the RDB database system. The latter was developed at RAND Organization by Walter W. Hobbs. The few Perl scripts included with NoSQL are still very close to the original RDB ones, so a good share of the credit for them goes to the original author.

Other major contributors to the original RDB system, besides Walter Hobbs, are:

• Chuck Bush
• Don Emerson
• Judy Lender
• Roy Gates
• Rae Starr

People who helped with turning RDB into NoSQL:

• Vincenzo (Vicky) Belloli
• David Frey
• Giuseppe Paternò
• Maurizio (Masar) Sartori
• Paul Lussier
• Seth LaForge
• Micah Stetson
• Thomas Miller
• Michael Somos
• Agustín Ferrin

The NoSQL logo was kindly provided by Kyle Hart.

As its name implies, NoSQL is not an SQL database but rather a shell-level tool, as explained in Philosophy of NoSQL. NoSQL data is contained in regular UNIX ASCII files, and so can be manipulated by regular UNIX utilities, e.g. ls, wc, mv, cp, cat, head, more, less, editors like 'vi,' etc., as well as by powerful versioning systems, such as RCS and CVS.

The form of each file of data is that of a relation, or table, with rows and columns of information.

To extract information, a file of data is fed to one or more "operators" via the UNIX Input/Output redirection mechanism.

There are also programs to generate, modify, and validate the data. A more through discussion of why this type of relational database structure makes sense is found in the book, "UNIX Relational Database Management", in the "References" section below.

It is assumed that the reader has at least a basic familiarity with the UNIX Operating System, including knowledge of Input/Output redirection (e.g., STDIN, STDOUT, pipes).

NoSQL tends to be biased in favor of Linux. This means that, wherever it matters, NoSQL makes use of the GNUversion of the various UNIX commands, as those are the ones normally found on Linux. NoSQL is Free Software, released under the terms of the GNU General Public License. As such, it fully qualifies also as Open Source Software.

---

What NoSQL is not

NoSQL has been around for more than a decade now and it has nothing to do with the newborn NoSQL Movement, which has been receiving much hype lately. While the former is a well-defined software package, is a relational database to all effects and just does intentionally not use SQL as a query language, the newcomer is mostly a concept (and by no means a novel one either), which departs from the relational model altogether and it should therefore have been called more appropriately "NoREL", or something to that effect.

---

How to get NoSQL

NoSQL can be freely downloaded from http://www.strozzi.it/shared/nosql/. Make sure you get also any patches to the base code, for the version that you download.

Starting from version 4.0.14 the NoSQL development repository was moved from CVS to Subversion. Authorized users can download the latest development code from my Subversion server:

svn checkout --username anonymous svn://subversion.strozzi.it/pub/nosql/trunk

Versions of NoSQL from 4.1.0 to 4.1.6 depend on the Shell-Ware Utilities (SWU) package, which can be downloaded from http://www.strozzi.it/shared/swu/ . Since only a small portion of SWU is actually required for NoSQL to work, starting from version 4.1.7 I have removed such dependency by including a minimal, stripped-down version of SWU with NoSQL itself. If you have already installed the full-blown SWU, NoSQL will use that one instead.

BOSS/GNU Debian Linux Configure Network Interface Cards – IP address and Netmasks

Q. How do I configure networking or network interface card on HP Debian Linux U1 Server?

A. Debian Linux provides GUI, command line tools and direct configuration file editing options to set up networking. Network configuration from the command line is possible.

Configure the Network Manually

You can use ip or ifconfig command to configure IP address and other information.

Task: Display the Current Network Configuration

Type the following command:
$ ip address show
Output:

1: lo:  mtu 16436 qdisc noqueue
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0:  mtu 1500 qdisc pfifo_fast qlen 100
    link/ether 00:19:d1:2a:ba:a8 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.1/24 brd 192.168.2.255 scope global eth0
    inet6 fe80::219:d1ff:fe2a:baa8/64 scope link
       valid_lft forever preferred_lft forever
3: ra0:  mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:17:9a:0a:f6:44 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.106/24 brd 192.168.1.255 scope global ra0
    inet6 fe80::217:9aff:fe0a:f644/64 scope link
       valid_lft forever preferred_lft forever
4: ppp0:  mtu 1496 qdisc pfifo_fast qlen 3
    link/ppp
    inet 10.1.3.103 peer 10.0.31.18/32 scope global ppp0

You can also use ifconfig -a command, enter:
$ ifconfig -a
Output:

eth0      Link encap:Ethernet  HWaddr 00:19:D1:2A:BA:A8
          inet addr:192.168.2.1  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::219:d1ff:fe2a:baa8/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:15819 errors:0 dropped:0 overruns:0 frame:0
          TX packets:27876 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:1695948 (1.6 MB)  TX bytes:40399983 (38.5 MB)
          Base address:0x1000 Memory:93180000-931a0000 

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:11943 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11943 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:7024449 (6.6 MB)  TX bytes:7024449 (6.6 MB)

ppp0      Link encap:Point-to-Point Protocol
          inet addr:10.1.3.103  P-t-P:10.0.31.18  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1496  Metric:1
          RX packets:34922 errors:0 dropped:0 overruns:0 frame:0
          TX packets:15764 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:3
          RX bytes:50535608 (48.1 MB)  TX bytes:1256881 (1.1 MB)

ra0       Link encap:Ethernet  HWaddr 00:17:9A:0A:F6:44
          inet addr:192.168.1.106  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::217:9aff:fe0a:f644/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:73809 errors:0 dropped:0 overruns:0 frame:0
          TX packets:31332 errors:1 dropped:1 overruns:0 carrier:0
          collisions:27 txqueuelen:1000
          RX bytes:61373519 (58.5 MB)  TX bytes:5007190 (4.7 MB)
          Interrupt:20

The information is grouped by network interfaces. Every interface entry starts with a digit, called the interface index, with the interface name displayed after the interface index. In the above example, there are four interfaces:

• lo : Loopback interface, used to access local services such as proxy or webserver http://127.0.0.1/
• eth0 : The first Ethernet interface connected to network switch or router
• ra0 : The first wireless interface
• ppp0 :The first point-to-point interface, used to connect via VPN or dial up service

Task: Device / Interface Statistics

Type the following command:
$ ip -s link show interface-name
$ ip -s link show eth0
$ ip -s link show ppp0
Output:

4: ppp0:  mtu 1496 qdisc pfifo_fast qlen 3
    link/ppp
    RX: bytes  packets  errors  dropped overrun mcast
    50537336   34946    0       0       0       0
    TX: bytes  packets  errors  dropped carrier collsns
    1257745    15776    0       0       0       0

Change the Current Network Configuration

You must login as the root to change current network settings.

Task: Assign an IP Address to a Device Interface

In the following example, the command assigns the IP address 192.168.1.10 to the device eth0. The network mask is 24 (255.255.255.0) bits long. The brd + option sets the broadcast address automatically as determined by the network mask.
# ip address add 192.168.1.100/24 brd + dev eth0
You can also use ifconfig command, enter
# ifconfig eth0 192.168.1.100 netmask 255.255.255.0 up

Task: Remove / Delete / Deactivate IP address from a device interface

To remove IP / delete device, enter:
# ip address del 192.168.1.100 dev eth0
OR
# ifconfig eth0 down

Save Network Settings to a Configuration File

To change the current network configuration setting you'll need to edit /etc/network/interfaces file using a text editor such as vi. This is the only way to save device setting to a configuration file so that system can remember changes after a reboot.

Task: Configure a Device Statically

Open /etc/network/interfaces file as the root user:
# vi /etc/network/interfaces
Let us assign static public routable (or private) IP address to eth0, enter:
auto eth0
iface eth0 inet static
address 192.168.2.1
netmask 255.255.255.0

Save and close the file. Where,

• auto eth0 : Identify the physical interfaces such as eth0, eth1 and so on
• iface eth0 inet static : This method used to define ethernet interfaces with statically allocated IPv4 addresses
• address 192.168.2.1 : Static IP address
• netmask 255.255.255.0 : Static netmask

Task: Configure a Device Dynamically with DHCP

Open /etc/network/interfaces file as the root user:
# vi /etc/network/interfaces
Let us configure eth0 using DHCP. When the device is configured by using DHCP, you don’t need to set any options for the network address configuration in the file.
auto eth0
iface eth0 inet dhcp
Save and close the file.
Where,

• auto eth0 : Identify the physical interfaces such as eth0, eth1 and so on
• iface eth0 inet dhcp : This method used to define ethernet interfaces with DHCP server allocated IPv4 addresses

Start and Stop Configured Interfaces

To apply changes to a configuration file, you need to stop and restart the corresponding interface
# /etc/init.d/networking stop
# /etc/init.d/networking start
# /etc/init.d/networking restart
You can also use following command to bring down or up the eth0. Disables the device eth0, enter:
# ifdown eth0
Enables eth0 again, enter:
# ifup eth0