| Page Item | Value |
| Title | Building a Search Engine Friendly Website |
| Description | Some advise for troubleshooting website search engine placement and subsequent search engine optimization |
| Keywords | search engine, placement, optimisation, optimization, troubleshooting, problems, issues, splash pages, cookies, javascript, html, validate, validator, validated, web robots, crawlers, crawling, web crawler, bots, robots |
| Robots Meta Tag | |
| Page Content | HOME | ABOUT ME | BIOTECHNOLOGY | ARTICLES | TOOLS | GALLERY | CONTACT Search: Go DEVELOPER TOOLS ASP Doc Tool ASP.NET Doc Tool SQL Doc Tool Index Server Companion The Website Utility TECHNICAL ARTICLES ASP ASP.NET JavaScript Transact SQL PHOTO GALLERIES Canon EOS 300D Samples Red Arrows 2004 Living Coasts Web Page Backgrounds More Galleries... NEW STUFF SQL Color Coder Canon EOS 300D Samples The Website Utility Search Engine Optimisation Build an ASP Search Engine My Tropical Fishtank Savings Other New Stuff... POPULAR STUFF Regular Expressions ASP Documentation Tool Index Server ASP JavaScript Ad Rotator LINKS Business Website ASPAlliance Articles
Home Articles Building a Search Engine Friendly Website Websites such as the ASPAlliance are full of useful articles about how to build your website using technologies such as ASP and ASP.NET. But even though you may have the most technically brilliant website around, it's going to be of limited use if the website isn't able to be crawled by the web robots that Google , Yahoo and the other search engines use to find content to add to their search catalogs. This article highlights some of the surprisingly common issues that affect search engine robots. The problems can be divided into two groups: Urgent Web Crawling Issues Search Engine Optimisation Issues (or wasted opportunities as I prefer to call them) . If you want to test your website for many issues, then an application I have created called The Website Utility can be used. It contains a built-in web crawler which can be used to test whether web robots can successfully navigate a website. It also reveals common website search engine optimization issues . Urgent Web Crawling Issues These are serious issues that will prevent a web crawling robot from indexing some or in a worst case scenario, all of a website's pages. Here are a few commonly encountered problems: Invalid HTML Web crawling robots may have difficulties navigating websites. These issues can be hard to spot, since most web browsers are tolerant of sloppy HTML. To resolve possible errors, try using an HTML validator such as the W3C's HTML validation service . You should pay particular care to ensuring that the website's internal hyperlinks are using valid HTML. For example, this is a valid link: a href= default.asp title= Home Home Page /a Whereas this link may not be recognised by a web crawling robot: a href=default.asp title=Home Home Page /a Robots.txt or Robots Meta Tag Issues Most web robots, will respect both a web server's robots.txt file as well as the 'robots' meta tag that may be present in individual pages. These specify which pages a robot is allowed to crawl and retrieve content from. If a website, or portions of that website cannot appear to be submitted successfully to search engines it is possible that the search engine is not accepting that website due to it being marked as non-retrievable in either the robots.txt file or in individual files through the use of the 'robots' meta tag. JavaScript Navigation Web crawling robots are getting more sophisticated, but they will almost certainly have problems following links that are only accessible from JavaScript. This is common on websites that have JavaScript-based drop-down menus as their main means of site navigation. It is, therefore, a good idea to ensure that the website is fully navigable without having to rely on JavaScript. Needless to say, providing alternative means of navigation will also assist human visitors using web browsers that do not support JavaScript, or those that have disabled JavaScript in their browsers. As well as JavaScript, this issue also applies to links that are only in widgets such as Java Applets and Macromedia Flash animations. Compulsory Cookies Although it is perfectly acceptable to use cookies on a website, care should be taken to avoid compulsory cookies (i.e. the website won't work if a specific cookie isn't set). The easiest way to check this is to browse the website with cookies disabled in the web browser. Some web crawling robots are also able to support session cookies, such as those used by ASP and ASP.NET. Search Engine Optimisation Deficiencies Missing Page Titles Missing page titles is a common search engine optimisation deficiency. Since many search engines place significant weighting on a page's title, it is essential to ensure all pages have a title. Duplicated Page Titles Alongside missing page titles, duplicate page titles is another common issue. The report will show each duplicated title and which pages share the specific title. A surprising number of websites use the same page title every single page in the site, which is in most instances an obvious waste of potential site optimisation for search engines. Missing/Blank Description and Keywords Meta Tags Although most search engines now give much less weighting to the content of description and keywords meta tags than they once did, it is still a good idea to include them. In particular, the description meta tag is often used as a page's summary in search engine results pages. It is also worth including them if your website has its own search facility based on Microsoft's Index Server or The Website Utility's ASP Search Engine or JavaScript Search Engine . Insufficient Indexable Content Search engines are still mostly based on the retrieval and indexing of text content on web pages. Consequently, it is important to ensure there is plenty of text on . Although file size is often an indication of the amount of indexable text on a page, this may also indicate that the page contains a lot of non-indexable content (e.g. HTML tags and JavaScript). Use of Splash Pages/Screens Splash screens are home pages that display the organisation's logo for a few seconds before redirecting to the main content page, or they contain a Macromedia Flash animation and a click to continue link below. Whilst splash screens look attractive (and keep web designers in business!) they result in human website visitors being an extra click away from the content they may be looking for. From a search engine placement point of view, splash screens can also adversely affect a site's search engines rankings as they mean the first (and most important) page in the website has very little indexable content on it, few links to other pages in the website and often use redirection to other pages. Testing Your Website A good way of testing to ensure your website is search engine friendly is to crawl it using a web crawler. The Website Utility is a Windows application that can be used to test the crawlability of websites. It contains a built-in web crawler which can be used to test whether web robots can successfully navigate a website. It also reveals common website search engine optimization issues and errors such as broken links . Finally, remember that testing a website for errors and search engine optimisation issues will almost certainly result in higher levels of satisfaction from the owner - useful if your livelihood depends on happy customers and repeat business! Useful Development Tools ASP Documentation Tool Automatically creates developer documentation for ASP 2.0 and 3.0 web applications written in VBScript and JScript. Documentation for Microsoft Access, SQL Server 7/2000 databases and Visual Basic 6.0 components associated with the web application can also be incorporated into the reports. Documentation is created in HTML, HTML Help and plain text formats. View Sample Output (HTML Help format). View Sample Output (HTML Format). Download Trial Version (5.2Mb ZIP file). ASP.NET Documentation Tool Automatically creates developer documentation for ASP.NET web applications written in C# or VB.NET. Documentation for SQL Server 7/2000 databases and C#/VB.NET components associated with the web application can also be incorporated into the reports. Documentation is created in HTML, HTML Help and plain text formats. View Sample Output (HTML Help format). View Sample Output (HTML Format). Download Trial Version (2.9Mb ZIP file). SQL Documentation Tool The SQL Documentation Tool creates technical documentation for Microsoft SQL Server 7.0 and 2000 databases. Technical documentation is created in HTML and HTML Help formats. The HTML Help format documentation is fully searchable and cross referenced. The SQL Documentation Tool documents SQL Server Tables, Views, Stored Procedures, Triggers and Table Relationships. View Sample Output (HTML Help format). View Sample Output (HTML Format). Download Trial Version (10.3Mb ZIP file). Index Server Companion The Index Server Companion is a Windows application that extends the functionality of Microsoft Index Server so that it is able to index content from remote websites and also from ODBC databases. As such it can be used as a low cost alternative to Site Server 3.0 Search. View Product Documentation (119K ZIP file). Try Sample Search Facility . Download Trial Version (1.7Mb ZIP file). The Website Utility The Website Utility examines websites for errors and areas that need to be optimised for search engines by using a built in web crawling engine. Errors checked for include broken or moved hyperlinks, missing page titles and missing meta tags. It also generates HTML for use in creating website site maps (table of contents pages - like this one ), and is able to create both client-side JavaScript Search Engines and server-side ASP Search Engines for a website. View Sample Output (HTML Help format). View Sample Output (HTML Format). Download Trial Version (3Mb ZIP file). Site Map All content is © 1995 - 2006 Brett Burridge |
| Image Alt Tags | Brettb.Com Microsoft Certified Professional View Sample Output (HTML Help format) View Sample Output (HTML Format) Download Trial Version View Sample Output (HTML Help format) View Sample Output (HTML Format) Download Trial Version View Sample Output (HTML Help format) View Sample Output (HTML Format) Download Trial Version View Product Documentation Try Sample Search Facility Download Trial Version View Sample Output (HTML Help format) View Sample Output (HTML Format) Download Trial Version ASP Documentation Tool - Free Trial Available! 1000MB and 40GB for $7.95 a month! |
| Internal Links | http://www.brettb.com/redirector.asp (12 links in this page) [ Robot View of this URL ] http://www.brettb.com/Website_Search_Engine_Optimisation.asp (3 links in this page) [ Robot View of this URL ] http://www.brettb.com/Default.asp (3 links in this page) [ Robot View of this URL ] http://www.brettb.com/technicalwriting.asp (2 links in this page) [ Robot View of this URL ] http://www.brettb.com/DeveloperTools.asp (2 links in this page) [ Robot View of this URL ] http://www.brettb.com/TheWebsiteUtility.asp (2 links in this page) [ Robot View of this URL ] http://www.brettb.com/ASPDocumentationTool.asp (2 links in this page) [ Robot View of this URL ] http://www.brettb.com/JavaScriptArticles.asp [ Robot View of this URL ] http://www.brettb.com/SearchResults.asp [ Robot View of this URL ] http://www.brettb.com/js_banner_ad_rotator.asp [ Robot View of this URL ] http://www.brettb.com/web.asp [ Robot View of this URL ] http://www.brettb.com/CanonEOS300D_Gallery1.asp [ Robot View of this URL ] http://www.brettb.com/BuildingAnASPSearchEngine.asp [ Robot View of this URL ] http://www.brettb.com/backgrounds.asp [ Robot View of this URL ] http://www.brettb.com/VBScriptRegularExpressions.asp [ Robot View of this URL ] http://www.brettb.com/toc.asp [ Robot View of this URL ] http://www.brettb.com/ASP.NETArticles.asp [ Robot View of this URL ] http://www.brettb.com/Investments_ISAs.asp [ Robot View of this URL ] http://www.brettb.com/TransactSQLColorCoder.asp [ Robot View of this URL ] http://www.brettb.com/MyTropicalFishtank.asp [ Robot View of this URL ] http://www.brettb.com/CanonEOS300D_Gallery3.asp [ Robot View of this URL ] http://www.brettb.com/ASPWatchArticles.asp [ Robot View of this URL ] http://www.brettb.com/Red_Arrows_2004.asp [ Robot View of this URL ] http://www.brettb.com/Living_Coasts_Photos.asp [ Robot View of this URL ] http://www.brettb.com/SQL_Help.asp [ Robot View of this URL ] http://www.brettb.com/contact.asp [ Robot View of this URL ] http://www.brettb.com/Biotechnology.asp [ Robot View of this URL ] http://www.brettb.com/Gallery.asp [ Robot View of this URL ] http://www.brettb.com/ASPNetDocumentationTool.asp [ Robot View of this URL ] http://www.brettb.com/gallery.asp [ Robot View of this URL ] http://www.brettb.com/what's_new.asp [ Robot View of this URL ] http://www.brettb.com/IndexServerCompanion.asp [ Robot View of this URL ] http://www.brettb.com/SearchingIndexServerWithASP.asp [ Robot View of this URL ] |
Report generated by The Website Utility 2.8