How To Install python-w3lib on Debian 9

In this tutorial we learn how to install python-w3lib on Debian 9. python-w3lib is Collection of web-related functions for Python 2

Introduction

In this tutorial we learn how to install python-w3lib on Debian 9.

What is python-w3lib

python-w3lib is:

Python module with simple, reusable functions to work with URLs, HTML, forms, and HTTP, that aren??t found in the Python standard library.

This module is used to, for example:

  • remove comments, or tags from HTML snippets
  • extract base url from HTML snippets
  • translate entities on HTML strings
  • encoding mulitpart/form-data
  • convert raw HTTP headers to dicts and vice-versa
  • construct HTTP auth header
  • RFC-compliant url joining
  • sanitize urls (like browsers do)
  • extract arguments from urls

The code of w3lib was originally part of the Scrapy framework but was later stripped out of Scrapy, with the aim of make it more reusable and to provide a useful library of web functions without depending on Scrapy.

This is the Python 2 version of the package.

There are three methods to install python-w3lib on Debian 9. We can use apt-get, apt and aptitude. In the following sections we will describe each method. You can choose one of them.

Install python-w3lib Using apt-get

Update apt database with apt-get using the following command.

sudo apt-get update

After updating apt database, We can install python-w3lib using apt-get by running the following command:

sudo apt-get -y install python-w3lib

Install python-w3lib Using apt

Update apt database with apt using the following command.

sudo apt update

After updating apt database, We can install python-w3lib using apt by running the following command:

sudo apt -y install python-w3lib

Install python-w3lib Using aptitude

If you want to follow this method, you might need to install aptitude first since aptitude is usually not installed by default on Debian. Update apt database with aptitude using the following command.

sudo aptitude update

After updating apt database, We can install python-w3lib using aptitude by running the following command:

sudo aptitude -y install python-w3lib

How To Uninstall python-w3lib on Debian 9

To uninstall only the python-w3lib package we can use the following command:

sudo apt-get remove python-w3lib

Uninstall python-w3lib And Its Dependencies

To uninstall python-w3lib and its dependencies that are no longer needed by Debian 9, we can use the command below:

sudo apt-get -y autoremove python-w3lib

Remove python-w3lib Configurations and Data

To remove python-w3lib configuration and data from Debian 9 we can use the following command:

sudo apt-get -y purge python-w3lib

Remove python-w3lib configuration, data, and all of its dependencies

We can use the following command to remove python-w3lib configurations, data and all of its dependencies, we can use the following command:

sudo apt-get -y autoremove --purge python-w3lib

Dependencies

python-w3lib have the following dependencies:

References

Summary

In this tutorial we learn how to install python-w3lib package on Debian 9 using different package management tools: apt, apt-get and aptitude.