Introduction
A uniform resource locator (URL) is a reference to a resource that specifies the location of the resource on a computer network and a mechanism for retrieving it. A URL is a specific type of uniform resource identifier (URI), although many people use the two terms interchangeably. A URL implies the means to access an indicated resource, which is not true of every URI. URLs occur most commonly to reference web pages (http), but are also used for file transfer (ftp), email (mailto), database access (JDBC), and many other applications.
Most web browsers display the URL of a web page above the page in an address bar. A typical URL has the form http://www.example.com/index.html, which indicates the protocol type (http), the domain name, (www.example.com), and the specific web page (index.html).
History
The Uniform Resource Locator was standardized in 1994 by Tim Berners-Lee and the URI working group of the Internet Engineering Task Force (IETF) as an outcome of collaboration started at the IETF Living Documents “Birds of a Feather” session in 1992. The format combines the pre-existing system of domain names (created in 1985) with file path syntax, where slashes are used to separate directory and file names. Conventions already existed where server names could be prepended to complete file paths, preceded by a double-slash (//).
Berners-Lee later regretted the use of dots to separate the parts of the domain name within URIs, wishing he had used slashes throughout. For example,http://www.example.com/path/to/name would have been written http:com/example/www/path/to/name. Berners-Lee has also said that, given the colon following the URI scheme, the two slashes before the domain name were also unnecessary.
Syntax
Every HTTP URL consists of the following, in the given order. Several schemes other than HTTP also share this general format, with some variation.
- the scheme name (commonly called protocol, although not every URL scheme is a protocol, e.g. mailto is not a protocol)
- a colon, two slashes,
- a host, normally given as a domain name For example, http://www.example.com/path/to/name would have been written http:com/example/www/path/to/name but sometimes as a literal IP address
- optionally a colon followed by a port number
- the full path of the resource
The scheme says how to connect, the host specifies where to connect, and the remainder specifies what to ask for.
For programs such as Common Gateway Interface (CGI) scripts, this is followed by a query string, and an optional fragment identifier.
The syntax is:
- scheme://[user:password@]domain:port/path?query_string#fragment_id
Component details:
- The scheme, which in many cases is the name of a protocol (but not always), defines how the resource will be obtained. Examples include http, https, ftp, file and many others. Although schemes are case-insensitive, the canonical form is lowercase.
- The domain name or literal numeric IP address gives the destination location for the URL. A literal numeric IPv6 address may be given, but must be enclosed in [ ] e.g.[db8:0cec::99:123a].
- The domain google.com, or its numeric IP address 173.194.34.5, is the address of Google’s website.
- The domain name portion of a URL is not case sensitive since DNS ignores case:
- http://en.example.org/ and HTTP://EN.EXAMPLE.ORG/ both open the same page.
- The port number, given in decimal, is optional; if omitted, the default for the scheme is used.
- For example, http://vnc.example.com:5800 connects to port 5800 of vnc.example.com, which may be appropriate for a VNC remote control session. If the port number is omitted for an http: URL, the browser will connect on port 80, the default HTTP port. The default port for an https: request is 443.
- The path is used to specify and perhaps find the resource requested. This path may or may not describe folders on the file system in the web server. It may be very different from the arrangement of folders on the web server. It is case-sensitive, though it may be treated as case-insensitive by some servers, especially those based on Microsoft Windows.
- If the server is case sensitive and http://en.example.org/wiki/URL is correct, then http://en.example.org/WIKI/URL or http://en.example.org/wiki/url will display an HTTP 404 error page, unless these URLs point to valid resources themselves.
- The query string contains data to be passed to software running on the server. It may contain name/value pairs separated by ampersands, for example
- ?first_name=John&last_name=Doe.
- The fragment identifier, if present, specifies a part or a position within the overall resource or document.
- When used with HTML, it usually specifies a section or location within the page, and used in combination with Anchor elements or the “id” attribute of an element, the browser is scrolled to display that part of the page.
The scheme name defines the namespace, purpose, and the syntax of the remaining part of the URL. Software will try to process a URL according to its scheme and context. For example, a web browser will usually dereference the URL http://example.org:80 by performing an HTTP request to the host at example.org, using port number 80.
Other examples of scheme names include https, gopher, wais, ftp. URLs with https as a scheme (such as https://example.com/) require that requests and responses will be made over a secure connection to the website. Some schemes that require authentication allow a username, and perhaps a password too, to be embedded in the URL, for exampleftp://asmith@ftp.example.org. Passwords embedded in this way are not conducive to security, but the full possible syntax is
- scheme://username:password@domain:port/path?query_string#fragment_id
Other schemes do not follow the HTTP pattern. For example, the mailto scheme only uses valid email addresses. When clicked on in an application, the URL mailto:bob@example.commay start an e-mail composer with the address bob@example.com in the To field. The tel scheme is even more different; it uses the public switched telephone network for addressing, instead of domain names representing Internet hosts.
List of allowed URL characters
Unreserved
The alphanumerical upper and lower case character may optionally be encoded:
- A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
- a b c d e f g h i j k l m n o p q r s t u v w x y z
- 0 1 2 3 4 5 6 7 8 9 – _ . ~
Reserved
Special symbols must sometimes be percent-encoded:
- ! * ‘ ( ) ; : @ & = + $ , / ? % # [ ]
Further details can for example be found in RFC 3986 and http://www.w3.org/Addressing/URL/uri-spec.html.
Relationship to URI
A URL is a URI that, in addition to identifying a web resource, provides a means of locating the resource by describing its “primary access mechanism (e.g., its network location)”.
Internet hostnames
A hostname is a domain name assigned to a host computer. This is usually a combination of the host’s local name with its parent domain’s name. For example, en.example.org consists of a local hostname (en) and the domain name example.org. The hostname is translated into an IP address via the local hosts file, or the domain name system (DNS) resolver. It is possible for a single host computer to have several hostnames; but generally the operating system of the host prefers to have one hostname that the host uses for itself.
Any domain name can also be a hostname, as long as the restrictions mentioned below are followed. For example, both “en.example.org” and “example.org” can be hostnames if they both have IP addresses assigned to them. The domain name “xyz.example.org” may not be a hostname if it does not have an IP address, but “aa.xyz.example.org” may still be a hostname. All hostnames are domain names, but not all domain names are hostnames.
URL protocols
The protocol, or scheme, of a URL defines how the resource will be obtained. Two common protocols on the web are HTTP and HTTPS. For various reasons, many sites have been switching to permitting access through both the HTTP and HTTPS protocols. Each protocol has advantages and disadvantages, including for some of the users that one or the other protocol either does not function, or is very undesirable. When a link contains a protocol specifier it results in the browser following the link using the specified protocol regardless of the potential desires of the user.
Protocol-relative URLs
It is possible to construct valid URLs without specifying a protocol which are called protocol-relative links (PRL) or protocol-relative URLs. Using PRLs on a page permits the viewer of the page to visit new pages using whichever protocol was used to obtain the page containing the link. This supports continuing to use whichever protocol the viewer has chosen to use for obtaining the current page when accessing new pages.
An example of a PRL is //en.wikipedia.org/wiki/Main_Page which is created by removing the protocol prefix.
Internationalized URL
Internet users are distributed throughout the world using a wide variety of languages and alphabets. Users expect to be able to create URLs in their own local alphabets.
An internationalized resource identifier (IRI) is a form of URL that includes Unicode characters. All modern browsers support IRIs. The parts of the URL requiring special treatment for different alphabets are the domain name and path.
The domain name in the IRI is known as an internationalized domain name (IDN). Web and Internet software automatically convert the domain name into punycode usable by the Domain Name System.
For example, the Chinese web site http://見.香港 becomes the following for DNS lookup. xn-- indicates the character was not originally ASCII.
- http://xn--nw2a.xn--j6w193g/
The URL path name can also be specified by the user in the local alphabet. If not already encoded, it is converted to Unicode, and any characters not part of the basic URL character set are converted to English letters using percent-encoding.
For example, the following Japanese Web page http://domainname/引き割り.html becomes http://domainname/%E5%BC%95%E3%81%8D%E5%89%B2%E3%82%8A.html. The target computer decodes the address and displays the page.