Unknown Unknown: OSF

Showing posts with label OSF. Show all posts

Open Source Web Crawlers

I am back again with one more OpenSource list. Seeing the good response to my previous posts on OpenSource SNS Platforms and OpenSource CMS platforms, I am here presenting you with OpenSource WebCrawlers.

Before I go further let me tell you that WebCrawlers need not be for search engines, they can be used to have a mirror site, advanced web users can use them for working on some sites, people like me who like to browse sites off line can also use them.

The list:

Aperture : Aperture is a Java framework for extracting and querying full-text content and metadata from various information systems (e.g. file systems, web sites, mail boxes) and the file formats (e.g. documents, images) occurring in these systems.
Arachnid : Java based web spider framework.
Arale : Arale can download entire web sites or specific resources from the web.
Grub : Grub is a distributed internet crawler/indexer designed to run on multi-platform systems, interfacing with a central server/database.
Heritrix : Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.
HyperSpider : This collects the link structure of a website.
J-Spider : A highly configurable and customizable Web Spider engine.
JoBo : A basic web spider, with the main advantage over others being that it can automatically fill out forms (e.g. for automated login) and also use cookies for session handling.
Metis : Collect information from the content of web sites
Nutch : From the Apache stable.
OpenWebSpider : OpenWebSpider would be the base for a new Search engine developed from a community of opensource developers!
Spider : Spider is a complete standalone Java application designed to easily integrate varied datasources.
Web-Harvest : Collect desired Web pages and extract useful data from them.
WebEater : A Java program for web site retrieval and offline viewing
WebLeach : A fully featured website downloader/mirror tool in Java.
WebSphinx : A personal, Customizable crawler.
YaCy : Peer-Peer Web Search Engine

[lInks/sOurces]
Java Based Crawlers
More Java Based Crawlers
Crawl Track: Tracking the Crawlers
8Legs: OpenSource Spiders and Search Engines
Build A Web Spider on Linux
All About Search: Search Tools

Open Source CMS platforms

Following up the good response to the "Open Source Social Networking Platforms", I am back with a list of CMS platforms. If only CMS platforms were as simple as SNS platforms. There are way too many. Not very surprising considering the broad definition of content and the broad applications they are used in. I present here a small list of Open Source CMS platforms, without bothering about all the commercial ones.

At the end of this post, I provide a few links which give you more information than what I mention here.

Open Source

Alfresco
CPS
Drupal
Fedora
JetNuke
Joomla
Magnolia
Mambo
PHPNuke
Pligg
Plone
Plume
Roller
Sakai Project
SilverStripe
TextPattern
Umbraco (An Open Source CMS based on Microsoft's ASP.NET)

[lInks]
Open Source CMS (The name says it all, the best of all lists)
e-Learning (CMS for developing educational content)
Making a Better Open Source CMS
sex, lies and CMS Vendors (Clearing the Myths)

If you have used any of the above, please leave your experiences in the comments.

Open Source: Social Networking Platforms

Social networking is the in thing. It has transcended the realms of your friends. Now we have community sites for almost everything that you can think of. In India too we have lots of startups coming in this space.

Some of you with ideas in this space might be overwhelmed by the amount of work you have to do get your idea up and running. Don't worry, there are lot of good open source platforms out there.

If you feel Open source stuff is a bit taxing, then you have some hosted solutions. Just pay a few bucks and you have a full fledged social net running.

What are you waiting for pick your favorite and let your community grow.