I believe that there are two main reasons for its limited use, especially by economists.

• You need to transform your existing references into BibTeX format. If you want to add a reference to a paper you're writing and you're not using BibTeX, it's easy to either copy and paste the reference from another of your papers, or from some other source. To convert a reference to BibTeX format takes somewhat more work. A solution is to use text2bib, which is intended to convert references in almost any reasonable text format to BibTeX.
• You want a citation in your paper to look like "Nash (1950)" and you want a reference to look something like

  Nash, John F., Jr. (1950), "Equilibrium points in N-person games", Proceedings of the National Academy of Sciences of the United States of America 36, 48–49.

  The default BibTeX style produces a result that is not even close. Dozens of BibTeX styles are available, but my efforts to find one that produces citations and references like these were fruitless. A solution is to use te.bst (the style used by Theoretical Economics). [Since I posted the first version of this page, Christian Odendahl has pointed out a style with similar features. He notes that when using this style it is necessary to use \citet rather than \cite to produce a citation like "Nash (1950)".]

Using BibTeX has several significant benefits:

• You need to type each reference only once. Once it is in your database, it can be used in any file you write.
• The style of all your citations in any given paper will be consistent. If, for example, the style you use causes \cite{arrow72} to produce "Arrow (1972)", then you'll never find "Arrow [1972]" in your paper.
• If you cite more than one paper by the same author in the same year, BibTeX will append the appropriate letters to the years—you don't need to fiddle around with them and change "Arrow (1972)" to "Arrow (1972a)" when you add a reference to another paper by Arrow in 1972.
• You never need to fuss with the style of the list of references at the end of your paper. All you need to do is say \bibliography{research} and all the items you cite will be extracted from your research.bib bibliography file, ordered correctly, and formatted at the end of your paper according to the style file you specify.
• Every paper you cite will appear in the bibliography, and every paper in the bibliography will be one you cite (unless you explicitly specify otherwise)—without your having to do any manual check.

Create a bibliography database

A BibTeX database is a plain text file. You can find the full specification in LaTeX: A document preparation system by Leslie Lamport (ISBN 0-201-15790-X). The file should be given the extension bib.

Here are some ways to create a database: (1) convert references from text format using text2bib; (2) convert references in some other format using tools available on the net; (3) export references from Google Scholar into BibTeX after choosing BibTeX in the "Bibliography manager" section of your Scholar Preferences; (4) type the database from scratch.

However you create it, here's how it will look:

@article{ahu61,
       author={Arrow, Kenneth J. and Leonid Hurwicz and Hirofumi Uzawa},
       title={Constraint qualifications in maximization problems},
       journal={Naval Research Logistics Quarterly},
       volume={8},
       year = 1961,
       pages = {175-191}
     }

@book{ab94,
       author = {Charalambos D. Aliprantis and Kim C. Border},
       year = {1994},
       title = {Infinite Dimensional Analysis},
       publisher = {Springer},
       address = {Berlin}
     }

@incollection{m85,
       author={Maskin, Eric S.},
       year={1985},
       title={The theory of implementation in {N}ash equilibrium: a survey},
       booktitle={Social Goals and Social Organization},
       editor={Leonid Hurwicz and David Schmeidler and Hugo Sonnenschein},
       pages={173-204},
       publisher={Cambridge University Press},
       addess={Cambridge}
     }

Notes:

• The order of the fields is unimportant.
• The string following the item type (in the first example above, that's "ahu61") is a label that you use to refer to the item when you cite it. You can choose any label you want.
• Several types in addition to article, book, and incollection are available.
• For each types, some fields are mandatory and some are optional. As you would expect, the mandatory fields are more or less the minimal ones necessary to identify the item.
• Names: BibTeX interprets "Arrow, Kenneth J.", "Kenneth J. Arrow", and "Kenneth Joseph Arrow" to refer to an author whose last name is "Arrow" and whose forenames are "Kenneth J." or "Kenneth Joseph". So if you want to refer to an item by John Maynard Smith (whose last name was "Maynard Smith"), you need to format the name as "Maynard Smith, John" or alternatively "John {Maynard Smith}". [It's not exactly the case that BibTeX takes the string following the last space to be the last name. If that string is preceded by a string or strings that start with lowercase letters, those strings as well as the final string are treated as the last name. For example, if you write "Ernesto Guevara de la Serna", BibTeX interprets "de la Serna" to be the last name (which of course is wrong: you need to write "Guevara de la Serna, Ernesto).]
• An "and" appears between every pair of authors. (Of course, when the reference is formatted only the "ands" specified by the style appear.)
• Note that in the last reference, the first letter of "Nash" is enclosed in braces. That tells BibTeX that if a style calls for titles to use "sentence capitalization" (only the first letter of the title and the first letter of any proper nouns are capitalized) then the first letter of "Nash" should remain uppercase and not be lowercased
• Your bib file can contain references you don't cite. BibTeX will put in the list of references at the end of your paper only the ones that you cite (unless you explicitly tell it otherwise).

Create a LaTeX file

Your LaTeX file needs to include citations, a reference to the BibTeX style file you want to use, and a command to generate the list of references; in addition, some BibTeX style files require a companion LaTeX style file.

Here is an example, with the lines related to BibTeX highlighed.

\documentclass[12pt]{article}
\usepackage{natbib}

\begin{document}

\title{BibTeX in action}
\author{Martin J. Osborne}
\date{2008-1-13}

\maketitle

\section{Introduction}
This document illustrates the use of BibTeX\@.  You may want to refer to
\cite{ahu61} or \cite{ab94} or \cite{m85}.  Or you may want to cite a
specific page in a reference, like this: see \citet[p.~199]{m85}.  Or
perhaps you want to cite more than one paper by Maskin: \cite{m85, m99}.
Or you want to make a parenthetical reference to one or more articles, in
which case the \verb+\citealt+ in the \texttt{te.bst} bibliography style
omits the parentheses around the year (\citealt{ahu61}).

\bibliographystyle{te}

\bibliography{research}

\end{document}

The roles of the highlighted items are as follows.

• To start at the end, the last highlighted line tells BibTeX to create a list of references using those in the BibTeX file research.bib.
• The next-to-last line tells BibTeX to use the style file te.bst when formatting the references. You can of course use any style you want. To get this one, go to this page.
• The first highlighted line tells LaTeX to use a style that produces the right sort of citations in the text (matched with the format of the references produced by te.bst. You probably have this file already (assuming you have some implementation of TeX on your computer). If you don't, you can get it on this CTAN page.
• The remaining highlighted strings are citations.

When you run the LaTeX file through LaTeX and BibTeX (instructions below), you'll get output for the body of the document that looks roughly like this:

This document illustrates the use of BibTeX. You may want to refer to Arrow et al. (1961) or Aliprantis and Border (1994) or Maskin (1985). Or you may want to cite a specific page in a reference, like this: see Maskin (1985, p. 199). Or perhaps you want to cite more than one paper by Maskin: Maskin (1985, 1999). Or you want to make a parenthetical reference to one or more articles, in which case the \citealt in the te.bst bibliography style omits the parentheses around the year (Arrow et al. 1961).

A few more options for the \cite command are available. Here they are:

\citet{key}	Jones et al. (1990)
\citet*{key}	Jones, Baker, and Smith (1990)
\citep{key}	(Jones et al. 1990)
\citep*{key}	(Jones, Baker, and Smith 1990)
\citep[p.~99]{key}	(Jones et al., 1990, p. 99)
\citep[e.g.][]{key}	(e.g. Jones et al., 1990)
\citep[e.g.][p.~99]{key}	(e.g. Jones et al., 1990, p. 99)
\citeauthor{key}	Jones et al.
\citeauthor*{key}	Jones, Baker, and Smith
\citeyear{key}	1990
\citeapos{key}*	Jones et al.'s (1990)

*Assumes \citeapos is defined in your style or document like this:

\def\citeapos#1{\citeauthor{#1}'s (\citeyear{#1})}

(Thanks to Christopher M. Duncombe Rae for pointing out this simple way of generating a possessive citation.)

The list of references will look like this:

Aliprantis, Charalambos D. and Kim C. Border (1994), Infinite Dimensional Analysis. Springer, Berlin.

Arrow, Kenneth J., Leonid Hurwicz, and Hirofumi Uzawa (1961), "Constraint qualifications in maximization problems." Naval Research Logistics Quarterly, 8, 175–191.

Maskin, Eric S. (1999), "Nash equilibrium and welfare optimality." Review of Economic Studies, 66, 23–38.

Maskin, Eric S. (1985), "The theory of implementation in Nash equilibrium: a survey." In Social Goals and Social Organization (Leonid Hurwicz, David Schmeidler, and Hugo Sonnenschein, eds.), 173–204, Cambridge University Press.

To change the format, you can in principle edit the BibTeX style file, te.bst. However, the language in which this style file is written is arcane, and changes that are more than trivial are tricky. A better way to proceed is to create a new style file from scratch, using the custom-bib package (that's how I created te.bst). You run TeX on a file, which asks you a long list of questions about the features of the style you would like. You'll probably not be completely clear about your preferred answers to all the questions on your first attempt, but two or three runs should produce a format to your liking.

Run LaTeX and BibTeX

Finally, you need to run your file through LaTeX, then BibTeX, then a couple more times through LaTeX.

Typically, running LaTeX means typing latex <filename> at the command prompt and running BibTeX means typing bibtex <filename>. If you are using a program (like TeXnicCenter) with a graphical interface to TeX, you probably need to click on a button. If you are using BaKoMa TeX Word, you don't need to do anything to run LaTeX (it runs automatically); to get BibTeX to run you may need to click on the "Refresh document references ..." button (red arrows on a yellow background).