About This Research
Business owners rely on web-design firms to build their websites and help manage their online presence. Both owner and designer can evaluate the visual appearance of a site; the technical side of a site is a different matter. Here there is a large gap between the business owner’s knowledge and that of the designer. For the most part, technical elements are hidden from view. Though hidden, they have a great impact on a website’s health.
In this study we look at site health by examining a site’s component parts. When the components are in great shape, the site is “healthy.” A healthy site is more likely to rank well in search results, has improved click-through rates and delivers a better on-site experience for visitors. All these factors tend to generate increased leads and conversions and the added revenue those provide.
Just like for human physical health, insights come though examination and testing. Site auditing is a process to examine website health. Since few business owners audit the final work completed by a web designer, we decided to do it; to develop an overview of issues U.S. business owners should monitor, we decided to look at the work of top-ranked US web designers and evaluate the frequency and type of technical errors in websites they produce.
Report Sections
Site Health Score – By Designer
To determine if top ranked designers create technically healthy websites, we grouped sites by designer. Each designer had multiple sites included in the sample. An average health score was calculated from all sites by each designer.
Across all designers the average site health score is 73.5%. The range of site health scores for an individual designer’s portfolio goes from 49% to 91%. The lowest score for an individual site is 28% for a site by a designer in Lansing MI and the highest is 95% for a site by a designer in Wichita KS.
- Average Site Score 73.5%
- High Score 95%
- Low Score 28%
Total US Cities Included
Total Top Ranked Designers
Total Sites Checked
Having a top ranked designer is not a guarantee of a healthy site.
The wide range of results should concern business owners. Having a top ranked designer is not a guarantee of a healthy site. The results also identify an area of opportunity for designers who want to excel. Addressing the issues related to site health in all their projects would make their work exceptional as compared to others.
Many of the elements measured to formulate the site health score can dramatically affect organic search rankings and user experience. A business website not delivering desired results would benefit from a site audit and a tune-up to resolve these issues.
State Ranking
Parsing the data further, we looked to see if designers from any individual state produced noticeably better-quality websites. There was nearly a 20-point difference between the state with the lowest scores and the one with the highest. Montana and Alaska tied for the highest ranked scores at 83.5%, a solid 10 points above the national average. Maine ranked at the bottom at 65.0% a score that is 8.5 points below average.
There is a 20-point difference between the states with the highest scores and the one with the lowest.
- Average Site Score 73.5%
- High Score 83.5%
- Low Score 65%
Just over half of the states (26), scored better than average. States with small populations had fewer designers included in their state sample. Therefore, the scores of each designer had greater impact on the average in low population states. We address that issue later when we examine site health based on city population.
The top ten states are:
- Alaska and Montana (tie)
- North Dakota
- New Jersey
- Nebraska
- Missouri
- Colorado
- Washington DC
- Delaware and South Dakota (tie)
- Wisconsin
- Maryland
Site Health Score by City Size
Recognizing that population can distort state based ranking, we explored site health for US cities grouped by population instead of region or state.
Cities with a population between 121K-500K had health scores above the average. Those under 100K or over 500K in population had below average scores. The Top 6 largest cities had a score of 69.1% the lowest of any group.
About the Population Segments
Each population segment has roughly the same number of cities in it, except for the Top 6 group which has only the six largest cities.
Cities with a population between 121K-500K had health scores above average
Top Scoring City Size Group
Top designers in cities with a population between 120K-200K produced on average the healthiest sites: 2.1 points above the US average.
Average Site Health Score Based on City Size
- 43K-100K 73.1%
- 121K-200K 75.4%
- 201K-350K 74.4%
- 351K-500K 72.4%
- 501K-750K 72.4%
- 71K-1.25M 71.4%
- Top 6 69.1%
Severity of Issues Related to the Site Health Score
The tool SEMrush uses to identify issues, assigns each issue to one of three groups, each with a different level of severity.
- Errors – The most severe
- Warnings – Important to resolve
- Notices – useful information
Errors Identified
Errors are the issues of highest severity and have the most potential to improve or hinder a site’s placement in search results.
Over 62% of sites have one or more missing pages or other element (4xx).
About Duplication
Duplicate content, Title tags and Meta descriptions hinder clarity about a page’s unique purpose.
While the title tag and description may not directly contribute to ranking, they do provide clarity about page content and an opportunity to improve click through rates.
Almost 60% of sites studied had one or more pages with the same page title.
Most Frequent Errors
- Duplicate Content 67.5%
- 4xx Errors 62.2%
- Duplicate Title Tags 59.99%
- Pages not crawled 56.4%
- Duplicate Meta Descriptions 46.0%
- Broken Internal Links 44.5%
- Broken External Links 42.5%
- Issues with mixed content 27.4%
- Neither cononical URL or 301 redirect from HTTP homepage 19.2%
- Invalid sitemap.xml format 13.6%
- Incorrect pages found in sitemap.xml 12.5%
- Broken internal images 11.5%
Warnings Identified
The second level of issues identified by the crawl are warnings. They indicate items that are important to resolve to ensure your site is “clean” and ready to assist search engines in properly evaluating content.
About Links to HTTP Pages on HTTPS Site
The move toward secure connections to websites, even those that don’t involve purchasing online, has become a preferred practice. When the root of a site is secure (HTTPS) the images and links on every page should also be secure.
Over 96% of sites are heavy on code and light on readable text giving them a low text to HTML Ratio.
About Alt Descriptions
Alt descriptions provide a text reference for images on the site. This is important for visually impared persons and helps search engine crawlers identify images.
Over 34% of sites did not provide an sitemap to help guide search engine crawlers.
Most Frequent Warnings
- Low Text to HTML ratio 96.2%
- Low Word Count 86.6%
- Missing Meta Description 74.9%
- Uncompressed Pages 70.5%
- Missing ALT Descriptions 64.6%
- Missing h1 63.1%
- Title Element is Too Long 57.3%
- Sitemap.xml not found 34.8%
- Links Lead to HTTP Pages for HTTPS Site 34.2%
- missing hreflang andlang attributes 24.0%
- Temporary redirects 23.4%
Crawl Related Issues
For a search engine to index and rank a site, it must first crawl the site evaluating its content. Obstacles to that crawl can limit the length of the crawl and affect potential rank. Cleaning up crawl related errors helps search engine spiders do their job.
About 4XX Errors
4XX errors refers to the error number returned when trying to retrieve a page, image or other element. 404 errors, the most common, indicate that a page sought was not found to be present on the site.
When a site cannot be crawled accurately and efficiently, opportunity is lost
Help the Spiders
The automated tools used by search engines are referred to as spiders. No matter what you think about the arachnids in your neighborhood, you want these spiders to be your friend. Make their job easy so they will explore, analyze, index and rank your entire site.
Most Frequent Crawl Related Issues
- Uncompressed Pages 70.5%
- 4xx Errors 62.2%
- Pages not Crawled 56.4%
- Orphaned Sitemap Pages 36.2%
- Sitemap.xml not found 34.8%
- Blocked From Crawling 32.9%
- Slow Page Speed 19.4%
Content Related Issues
Good content is essential to success for any website. It is not about volume alone. Content must be relevant to your audience or customer, clear and compelling, and make a meaningful contribution to an idea, issue or question.
Thin Content
The term thin content refers to short posts or pieces of information that provide minimal value to the reader. They are sometimes created to target a keyword but offer nothing worthwhile to viewers.
When a site cannot be crawled accurately and efficiency opportunity is lost
About H1 Issues
A H1 tag identifies the top level heading on a page. Missing H1 tags are a lost SEO opportunity as they indicate what a page is about. Multiple H1 tags are a bit like when a grandma tells grandkids with competing pictures “all these pictures are the best”.
Most Frequent Content Related Issues
- Low Text to HTML Ratio 96.2%
- Low Word Count (Thin Content) 86.6%
- Duplicate Content 67.5%
- Missing ALT Attributes 64.6%
- Missing H1 63.1%
- Multiple H1 Tags 60.9%
- Duplicate Content in H1 and Title Tag 19.7%
Title and Meta Tag Related Issues
Page Titles and Meta tags help define what a page is about. Ever since meta, tiles, descriptions and keywords were deprecated as a ranking factor by search engines, many designers reduced or abandoned their use. They still have a role. Meta titles and meta descriptions give you an opportunity to inform a search engine what a page is about. They allow you to select the language that will be presented to the searching public, increasing the likelihood you will gain a click-through when your listing is shown.
46% of sites have duplicate meta descriptions; effectively providing no guidance as to how one page on the site is different from another.
Just the Right Title
Title tags are like book titles, they tell people (and search engines) what a page is about. Too long and they show poorly in search listings. Too short and they don’t convey relevant key terms. Duplicate titles say two or more pages are about the same thing. Missing titles says, “nothing to see here,” or “try to guess what its about.”
Most Frequent Title & Meta Tag Issues
- Missing Meta Description 74.9%
- Duplicate Title Tag 59.9%
- Title Element is Too Long 57.3%
- Duplicate Meta Descriptions 46.0%
- Title Element is Too Short 18.9%
- Title Tag is Missing or Empty 7.0%
Link Related Issues
Links are the connections between posts, and pages on the web. They are a currency of sort, as links to posts or pages convey an indication of value.
You Can't Get There from Here
Nofollow attributes have a legitimate place in some sites, but a broken link; internal or external, is a bridge that does not reach the other side of the canyon. Why would you send a customer or visitor that way?
Over 44% of sites have links pointing to a resource or page on their site that are broken
Broken Internal Images
A picture says a thousand words, but a broken internal image says nothing, except perhaps: “oops!” Over 11% of sites have broken internal images
Most Frequent Link Related Issues
- Broken Internal Links 44.5%
- Broken External Links 42.5%
- Nofollow attributes in External Links 24.7%
- Temporary Redirects 23.4%
- Nofollow attributes in Internal Links 19.8%
- Broken Internal Images 11.5%
Google Analytics
While there are other options for tracking traffic on a website, Google Analytics maintains a dominant position for US based websites. According to BuiltWith.com, 30.48% of US .com websites use Google Analytics with 15.85% using Analytics Classics and 15.64% using Universal Analytics (the more robust option).
Our sample of websites by top US designers showed the following use of Google Analytics. Top designers included Universal Analytics at a much higher rate than average. A good indicator.
- Universal Analytics 57.1%
- Analytic Classic 11%
- Tag Manager 2.3%
%
Sites with No Google Analytics Detected
Top 5 Take-A-Ways
One of the unique aspects of this study was the deliberate effort to explore the work quality of top ranking US web designers. The sites evaluated were created by people who know how to get their site ranked well in search engines, not by a friend, a self-taught individual or a low ranked web designer. The results showed that perfection is elusive, even for professionals.
Hiring a top ranked web designer is not a guarantee you will get a healthy site. Even good designers miss things.
There is tremendous variation in portfolio strength among highly rated designers. The lowest average score for a designer was 49% and the highest was 91%.
Quality sites can be developed anywhere in the country. Developers in cities with a population of between 121K-200K created the healthiest sites in our study.
Even the most basic critical things go wrong. Broken internal links were found in 44.5% of sites created by the best developers. Missing pages or resources affected 62.2% of all sites tested
There is lots of room for improved content. 86.6% of tested site had low word counts and 67.5% had duplicate content issues.
Final Thoughts for Business Owners
Sometimes problems with site quality begin because the developer gave us what we asked for and we did not know what to ask for. In many cases, quality issues arise from having the attitude that for a website it is okay to “set it and forget it”. Maintenance and updates might seem like a waste of money. Considering the errors identified in this study here are a few considerations for business owners.
When working with a developer, ask how they will audit the site for technical errors and make corrections. Ask for reports or have independent audits conducted. It may add to cost but it will inform you of the state of your site.
Just because a site looks beautiful does not mean it is healthy. Maybe the reason you don’t get leads you desire from your website is related to what is going on under the hood.
It is important to move past the idea that you can pay for your website once and then you’re done. In many ways websites are like a car. Fuel (content) and regular maintenance are needed for it to perform its function well. A designer offering a solid maintenance program including annual audits is an asset.
Yes, there are a lot of SEO and web marketing scams. However, for your site to make the best contribution it can to your business objectives, consistent attention needs to be paid to SEO.
The online world changes rapidly and new challenges and opportunities arise every year for businesses to get in front of the right customer. Capitalizing on those opportunities is worth the investment of time, attention and resources.
Content is critical. It is also hard for most people to create quality content. Don’t ignore it or settle for a weak substitute to “get it done”.
Final Thoughts for Web Designers
Perfection is elusive. Sometimes all it takes is to be the best in your area is to be 5% better than the competition. Your clients will encounter big challenges trying to reach and get response from their target audience. Give them all the help you can.
When working with a client, make site audits for technical errors a standard part of the process, not an add-on that a cost-cutting client will skip.
Beautiful design work should be backed up by rock solid technical excellence. Make sure the backend matches the front end in quality.
Plan sites to include unique titles and meta descriptions based on page content and designed for click through. Yes, they don’t add to rank but click through sure does.
Make sure that any hosting and maintenance plan you offer clients includes an annual check-up to catch any new errors and developments. A maintenance plan is not an “upsell” it is a chance to excel.
If a client has you configure Google analytics as part of their project, be sure to create goals related to the primary intention of the site: for example, sales transactions or form completions.
Research Methodology
Using anonymized google searches we identified the top organically ranked web designers and firms for 131 US cities (>100K pop). For states without multiple cities over 100K we sampled the next largest cities.
On each designer’s website, we randomly selected sites to sample from their portfolio of client sites. A total of 1020 sites from 407 designers are included in the study.
Using the site audit tool, part of the suite of SEO tools offered by our data partner SEMrush, each of the sample sites was crawled and the resulting data subjected to a technical analysis looking at 73 different factors. The site crawl made 22,181,269 individual checks across the sample of websites. Each site was also checked for the presence of an analytics package using GA Checker and BuiltWith.
Designer data was matched with US Census maps and data to provide geographic and population reference points.
Research Partners
Chris Weickert – Research Author – A 14 year veteran of B2C niche ecommerce, Chris is co-founder of EC Weickert, Inc. a US based web and online media company. Among other business activities he works as a business owner consultant in the area of website effectiveness. Chris coordinated the research project and authored its final report.
SEMrush – Research Data Partner – SEMrush is a leading competitive research service for online marketing. With a global reach and workforce in four countries SEMrush provides an all-in-one digital marketing suite used by thousands of SEO, and other digital marketing experts. SEMrush provided site error data on the study samples sites using their site audit tool.
BuiltWith – Research Data Partner – BuiltWith is an internet services company based in Manly, Australia. BuiltWith recognizes more than 14,000 web technologies and indexes a quarter of a billion websites to let people see which sites use which hosting company, content management system, or analytics program. BuiltWith provided technology reference points for study sample sites, especially related to site analytics.
Center for Governmental Studies, NIU – Research staff at Northern Illinois University’s Center for Governmental Studies provided assistance in determining sample size and configuration.
Appendix A: State Rankings – Average Site Health
Rank State Average
1 AK 83.5
2 MT 83.5
3 ND 80.5
4 NJ 79.0
5 NE 78.6
6 MO 77.5
7 CO 77.3
8 DC 76.9
9 DE 76.8
10 SD 76.8
11 WI 76.5
12 MD 76.4
13 NV 76.1
14 ID 76.0
15 WV 75.8
16 FL 75.3
17 VA 75.3
Rank State Average
18 RI 75.1
19 KS 74.9
20 OK 74.8
21 OH 74.6
22 NH 74.5
23 IA 74.4
24 MN 73.8
25 AZ 73.5
26 SC 73.5
27 IN 73.3
28 TX 73.3
29 VT 73.3
30 TN 73.0
31 CT 72.8
32 UT 72.8
33 LA 72.6
34 NC 72.4
Rank State Average
35 HI 72.3
36 WA 71.9
37 CA 71.8
38 AL 71.8
39 MI 71.5
40 NM 71.3
41 MA 71.2
42 PA 71.2
43 MS 70.8
44 IL 70.6
45 GA 70.2
46 WY 69.0
47 OR 68.4
48 NY 68.3
49 KY 67.5
50 AR 67.0
51 ME 65.0