$(wildcard) explained can be used anywhere in a Makefile or rule to get a list of files matching one or more glob style patterns.
$(wildcard) explained can be used anywhere in a Makefile or rule to get a list of files matching one or more glob style patterns.
For example, $(wildcard *.foo) returns a list of files ending in .foo. Recall that a list is a string where list elements are separated by spaces, so $(wildcard *.foo) might return a.foo b.foo c.foo. (If a filename itself contains a space then the returned list may appear incorrect because there's no way to spot the different between the list separator (a space) and then space in a filename.)
$(wildcard) can be called with a list of patterns, so $(wildcard *.foo *.bar) returns all the files ending in .foo or .bar. The $(wildcard) operator supports the following 'globbing' operators: * (match 0 or more characters), ? (match 1 character), and [...] (matches characters [123] or a range of characters [a-z]).
Another useful feature of $(wildcard) is that if the filename passed to it does not contain a pattern then that file is simply checked for existence. If the file exists then its name is returned, otherwise $(wildcard) returns an empty string. Thus $(wildcard) can be combined with $(if) to create an if-exists function:
if-exists = $(if ($wildcard $1),$2,$3)
which will print a.foo is there if a.foo exists, or a.foo is not there if not.
Some unexpected resultsEach of these examples use two variables for obtaining a list of files ending in .foo in a particular directory: WILDCARD_LIST and LS_LIST return the list of files ending in .foo by calling $(wildcard) and $(shell ls). The variable DIRECTORY holds the directory in which the examples look for files; for the current directory DIRECTORY is left empty.
The starting Makefile looks like this:
WILDCARD_LIST = wildcard returned \'$(wildcard $(DIRECTORY)*.foo)\'
LS_LIST = ls returned \'$(shell ls $(DIRECTORY)*.foo)\'
.PHONY: all
all:
@echo $(WILDCARD_LIST)
@echo $(LS_LIST)
With a single file a.foo in the current directory running make results in:
wildcard returned 'a.foo'
ls returned 'a.foo'
Now extend the Makefile so that it makes a file called b.foo using touch (I've skipped the declarations of WILDCARD_LIST and LS_LIST here since they are identical):
all: b.foo
@echo $(WILDCARD_LIST)
@echo $(LS_LIST)
b.foo:
@touch $@
Running this Makefile through make (with just the preexisting a.foo file) results in the following surprising output:
wildcard returned 'a.foo'
ls returned 'a.foo b.foo'
The ls is returning the correct list (since b.foo has been created by the time the all rule runs), but $(wildcard) is not; $(wildcard) appears to be showing the state before b.foo was created. Working with the .foo files in a subdirectory (not in the current working directory) results in different output. Here I've updated the Makefile so that it uses the $(DIRECTORY) variable to specify the subdirectory subdir. Once again there's a single preexisting file subdir/a.foo and the Makefile will create subdir/b.foo.
DIRECTORY=subdir/
.PHONY: all
all: $(DIRECTORY)b.foo
@echo $(WILDCARD_LIST)
@echo $(LS_LIST)
$(DIRECTORY)b.foo:
@touch $@
Running this Makefile results in:
ls returned 'subdir/a.foo subdir/b.foo'
Here both $(wildcard) and ls return the same results and both show the presence of the two .foo files: subdir/a.foo which existed before make was run and subdir/b.foo that was created by the Makefile. One final Makefile before getting down to an explanation of what's happening. In this Makefile $(warning) is used to print out a list of the .foo files that already exist in the subdirectory:
DIRECTORY=subdir/
$(warning Preexisting file: $(WILDCARD_LIST))
.PHONY: all
all: $(DIRECTORY)b.foo
@echo $(WILDCARD_LIST)
@echo $(LS_LIST)
$(DIRECTORY)b.foo:
@touch $@
And here's the output:
wildcard returned 'subdir/a.foo'
ls returned 'subdir/a.foo subdir/b.foo'
Notice now that make appears to be behaving like the first example above, the subdir/b.foo file that was made by the Makefile is invisible to $(wildcard) and doesn't appear even though it was created and ls found it.
Why This Happens
The unexpected, and apparently inconsistent, results above are because GNU Make contains its own cache of directory entries. $(wildcard) reads from that cache (and not directly from disk like ls) to get its results. When that cache is filled is vital to understanding the results the $(wildcard) will return.
GNU Make only fills the cache when it has too (for example, when it needs to read the directory entries to satisfy a $(wildcard) or other globbing request). If you know that GNU Make only fills the cache when needed then it's possible to explain the results seen above.
In the first example, GNU Make fills the cache for the current working directory when it starts. Hence the file b.foo doesn't appear in the output of $(wildcard) because it wasn't present when the cache was filled. In the second example, GNU Make hasn't filled the cache with entries from subdir until they were needed. The entries were first needed for the $(wildcard) which is performed after subdir/b.foo is created and hence subdir/b.foo does appear in the $(wildcard) output. In the final example, the $(warning) happens at the start of the Makefile which filled the cache (because it did a $(wildcard)) and hence subdir/b.foo was missing from the output of $(wildcard) for the duration of that make.
Predicting when the cache will be filled is very hard. $(wildcard) will fill the cache, but so will use of a globbing operator like * in the target or prerequisite list of a rule. Consider this example Makefile that builds two files (subdir/b.foo and subdir/c.foo) and does a couple of $(wildcard)'s:
DIRECTORY=subdir/
.PHONY: all
all: $(DIRECTORY)b.foo
@echo $(WILDCARD_LIST)
@echo $(LS_LIST)
$(DIRECTORY)b.foo: $(DIRECTORY)c.foo
@touch $@
@echo $(WILDCARD_LIST)
@echo $(LS_LIST)
$(DIRECTORY)c.foo:
@touch $@
The output may surprise you:
wildcard returned 'subdir/a.foo subdir/c.foo'
ls returned 'subdir/a.foo subdir/c.foo'
wildcard returned 'subdir/a.foo subdir/c.foo'
ls returned 'subdir/a.foo subdir/b.foo subdir/c.foo'
Even though the first $(wildcard) is being done in the rule that makes subdir/b.foo and after the touch that created subdir/b.foo there's no mention of subdir/b.foo in the output. Nor is there mention of subdir/b.foo in the output of the ls.
That's because the complete block of commands is expanded into its final form before any of the lines in the rule are run. So the $(wildcard) and $(shell ls) are done before the touch has run.
The output of $(wildcard) is even more unpredictable if the make is run in parallel with the -j switch. In that case the exact order in which the rules are run is not predictable and hence the output of $(wildcard) can be even less predictable.
Recommendations
Don't use $(wildcard) in a rule; only use $(wildcard) in the Makefile at parsing time (i.e. before any rules start running). If you restrict use of $(wildcard) to parsing time you can be assured of consistent results: $(wildcard) will show the state of the filesystem before make was run.
User Comments
Thank you for assisting me here; I have gained a lot of knowledge from reading your writings. I'm new to this site, and it's great to meet everyone.
Before any of the lines in the rule are run, the whole block of commands is put together into its final form. So the $(wildcard) and $(shell ls) are done before the touch.