All the mail mirrored from lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCH v7 0/3] Enable OpenCL support in Mesa 3D
@ 2018-05-04 11:46 Valentin Korenblit
  2018-05-04 11:46 ` [Buildroot] [PATCH v7 1/3] package/libclc: new package Valentin Korenblit
                   ` (2 more replies)
  0 siblings, 3 replies; 6+ messages in thread
From: Valentin Korenblit @ 2018-05-04 11:46 UTC (permalink / raw
  To: buildroot

This series is the continuation of the series "llvm for mesa3d".


Changes v6 -> v7:

*libclc headers installed to /usr/share/clc
*Remove libclc dependencies on target llvm
*Add OpenCL support for RadeonSI

Valentin Korenblit (3):
  package/libclc: new package
  package/mesa3d: enable OpenCL support
  package/clinfo: new package

 DEVELOPERS                                      |  2 ++
 package/Config.in                               |  2 ++
 package/clinfo/Config.in                        |  9 ++++++
 package/clinfo/clinfo.hash                      |  2 ++
 package/clinfo/clinfo.mk                        | 23 ++++++++++++++
 package/libclc/Config.in                        | 10 ++++++
 package/libclc/libclc.hash                      |  3 ++
 package/libclc/libclc.mk                        | 42 +++++++++++++++++++++++++
 package/mesa3d/0003-set-LIBCLC_INCLUDEDIR.patch | 26 +++++++++++++++
 package/mesa3d/Config.in                        |  9 ++++++
 package/mesa3d/mesa3d.mk                        |  9 ++++++
 11 files changed, 137 insertions(+)
 create mode 100644 package/clinfo/Config.in
 create mode 100644 package/clinfo/clinfo.hash
 create mode 100644 package/clinfo/clinfo.mk
 create mode 100644 package/libclc/Config.in
 create mode 100644 package/libclc/libclc.hash
 create mode 100644 package/libclc/libclc.mk
 create mode 100644 package/mesa3d/0003-set-LIBCLC_INCLUDEDIR.patch

-- 
2.14.3

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Buildroot] [PATCH v7 1/3] package/libclc: new package
  2018-05-04 11:46 [Buildroot] [PATCH v7 0/3] Enable OpenCL support in Mesa 3D Valentin Korenblit
@ 2018-05-04 11:46 ` Valentin Korenblit
  2018-05-28 20:33   ` Thomas Petazzoni
  2018-05-04 11:46 ` [Buildroot] [PATCH v7 2/3] package/mesa3d: enable OpenCL support Valentin Korenblit
  2018-05-04 11:46 ` [Buildroot] [PATCH v7 3/3] package/clinfo: new package Valentin Korenblit
  2 siblings, 1 reply; 6+ messages in thread
From: Valentin Korenblit @ 2018-05-04 11:46 UTC (permalink / raw
  To: buildroot

This patch provides libclc, an open source implementation of the
library requirements of the OpenCL C programming language, as
specified by the OpenCL 1.1 Specification. It is intended to be used
with Mesa Clover.

It needs to be compiled with host-clang, as it generates LLVM IR bitcode
files containing device builtin functions for each target.

Currently, libclc supports AMDGCN, R600 and NVPTX targets.

As OpenCL kernels can be built dynamically on the target using libclang and
libLLVM, it is necessary to have clc headers on the target. Buildroot removes
/usr/include in its target-finalize step, so clc headers are installed to
/usr/share/clc.

Given that clc headers are being installed to a non-standard location, it is
necessary to specify this path in Mesa's configure.ac. Otherwise, pkg-config
will output the absolute path to these headers located in STAGING_DIR, which
will cause a runtime error when calling clBuildProgram.

Signed-off-by: Valentin Korenblit <valentin.korenblit@smile.fr>
---
 DEVELOPERS                                      |  1 +
 package/Config.in                               |  1 +
 package/libclc/Config.in                        | 10 ++++++
 package/libclc/libclc.hash                      |  3 ++
 package/libclc/libclc.mk                        | 42 +++++++++++++++++++++++++
 package/mesa3d/0003-set-LIBCLC_INCLUDEDIR.patch | 26 +++++++++++++++
 6 files changed, 83 insertions(+)
 create mode 100644 package/libclc/Config.in
 create mode 100644 package/libclc/libclc.hash
 create mode 100644 package/libclc/libclc.mk
 create mode 100644 package/mesa3d/0003-set-LIBCLC_INCLUDEDIR.patch

diff --git a/DEVELOPERS b/DEVELOPERS
index 24d134cb70..3f5775ff48 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -1930,6 +1930,7 @@ F:	package/tstools/
 
 N:	Valentin Korenblit <valentin.korenblit@smile.fr>
 F:	package/clang/
+F:	package/libclc/
 F:	package/llvm/
 
 N:	Vanya Sergeev <vsergeev@gmail.com>
diff --git a/package/Config.in b/package/Config.in
index fe36d31867..dc0ee8a965 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -285,6 +285,7 @@ comment "Graphic libraries"
 	source "package/fbv/Config.in"
 	source "package/freerdp/Config.in"
 	source "package/imagemagick/Config.in"
+	source "package/libclc/Config.in"
 	source "package/linux-fusion/Config.in"
 	source "package/lite/Config.in"
 	source "package/mesa3d/Config.in"
diff --git a/package/libclc/Config.in b/package/libclc/Config.in
new file mode 100644
index 0000000000..4bba6a9175
--- /dev/null
+++ b/package/libclc/Config.in
@@ -0,0 +1,10 @@
+config BR2_PACKAGE_LIBCLC
+	bool "libclc"
+	depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+	depends on BR2_HOST_GCC_AT_LEAST_4_8
+	help
+	  libclc is an open source, BSD licensed implementation of
+	  the library requirements of the OpenCL C programming language,
+	  as specified by the OpenCL 1.1 Specification.
+
+	  http://libclc.llvm.org/
diff --git a/package/libclc/libclc.hash b/package/libclc/libclc.hash
new file mode 100644
index 0000000000..c5bf5a668c
--- /dev/null
+++ b/package/libclc/libclc.hash
@@ -0,0 +1,3 @@
+# locally calculated
+sha256 e2f1f294f93695f2d1f87ccb9760231b435702dd0c27abeb38baa97186674d9e  libclc-00236279a293b3737dee08c14f25923a889d2795.tar.gz
+sha256 c66aa55d8478b45f1018500af9f433dd271444ad304d683dd49ff2a42f30dad1  LICENSE.TXT
diff --git a/package/libclc/libclc.mk b/package/libclc/libclc.mk
new file mode 100644
index 0000000000..b87f191eb5
--- /dev/null
+++ b/package/libclc/libclc.mk
@@ -0,0 +1,42 @@
+################################################################################
+#
+# libclc
+#
+################################################################################
+
+# There are only two releases: release_35 and release_38, but the last
+# commit is from 2 years ago. Master has some recent activity.
+LIBCLC_VERSION = 00236279a293b3737dee08c14f25923a889d2795
+LIBCLC_SITE = https://git.llvm.org/git/libclc
+LIBCLC_SITE_METHOD = git
+LIBCLC_LICENSE = NCSA
+LIBCLC_LICENSE_FILES = LICENSE.TXT
+
+LIBCLC_DEPENDENCIES = host-clang host-llvm
+LIBCLC_INSTALL_STAGING = YES
+
+# C++ compiler is used to build a small tool (prepare-builtins) for the host.
+# It must be built with the C++ compiler from the host
+LIBCLC_CONF_OPTS = --with-llvm-config=$(HOST_DIR)/usr/bin/llvm-config \
+	--prefix="/usr" \
+	--includedir="/usr/share" \
+	--pkgconfigdir="/usr/lib/pkgconfig" \
+	--with-cxx-compiler=$(HOSTCXX)
+
+define LIBCLC_CONFIGURE_CMDS
+	(cd $(@D); $(TARGET_CONFIGURE_OPTS) ./configure.py $(LIBCLC_CONF_OPTS))
+endef
+
+define LIBCLC_BUILD_CMDS
+	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D)
+endef
+
+define LIBCLC_INSTALL_TARGET_CMDS
+	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D) DESTDIR=$(TARGET_DIR) install
+endef
+
+define LIBCLC_INSTALL_STAGING_CMDS
+	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D) DESTDIR=$(STAGING_DIR) install
+endef
+
+$(eval $(generic-package))
diff --git a/package/mesa3d/0003-set-LIBCLC_INCLUDEDIR.patch b/package/mesa3d/0003-set-LIBCLC_INCLUDEDIR.patch
new file mode 100644
index 0000000000..b3f9940c85
--- /dev/null
+++ b/package/mesa3d/0003-set-LIBCLC_INCLUDEDIR.patch
@@ -0,0 +1,26 @@
+From b1fdc84dff0f4c232eb58dded1dd6e3408ea5630 Mon Sep 17 00:00:00 2001
+From: Valentin Korenblit <valentin.korenblit@smile.fr>
+Date: Fri, 4 May 2018 10:24:58 +0200
+Subject: [PATCH] set LIBCLC_INCLUDEDIR
+
+Signed-off-by: Valentin Korenblit <valentin.korenblit@smile.fr>
+---
+ configure.ac | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/configure.ac b/configure.ac
+index a6a521794e..9e1dfef06f 100644
+--- a/configure.ac
++++ b/configure.ac
+@@ -2327,7 +2327,7 @@ if test "x$enable_opencl" = xyes; then
+                     PKG_CONFIG_PATH environment variable.
+                     By default libclc.pc is installed to /usr/local/share/pkgconfig/])
+     else
+-        LIBCLC_INCLUDEDIR=`$PKG_CONFIG --variable=includedir libclc`
++        LIBCLC_INCLUDEDIR="/usr/share"
+         LIBCLC_LIBEXECDIR=`$PKG_CONFIG --variable=libexecdir libclc`
+         AC_SUBST([LIBCLC_INCLUDEDIR])
+         AC_SUBST([LIBCLC_LIBEXECDIR])
+-- 
+2.14.3
+
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [Buildroot] [PATCH v7 2/3] package/mesa3d: enable OpenCL support
  2018-05-04 11:46 [Buildroot] [PATCH v7 0/3] Enable OpenCL support in Mesa 3D Valentin Korenblit
  2018-05-04 11:46 ` [Buildroot] [PATCH v7 1/3] package/libclc: new package Valentin Korenblit
@ 2018-05-04 11:46 ` Valentin Korenblit
  2018-05-04 11:46 ` [Buildroot] [PATCH v7 3/3] package/clinfo: new package Valentin Korenblit
  2 siblings, 0 replies; 6+ messages in thread
From: Valentin Korenblit @ 2018-05-04 11:46 UTC (permalink / raw
  To: buildroot

This patch provides Clover, the OpenCL 1.1 API implementation by Mesa
for AMD GPUs. It generates libOpenCL.so.

Add --disable-opencl-icd because in Mesa 18 it defaults to on.
When disabled, the shared library is named libOpenCL instead
of libMesaOpenCL and CL headers are installed.

Signed-off-by: Valentin Korenblit <valentin.korenblit@smile.fr>
---
 package/mesa3d/Config.in | 9 +++++++++
 package/mesa3d/mesa3d.mk | 9 +++++++++
 2 files changed, 18 insertions(+)

diff --git a/package/mesa3d/Config.in b/package/mesa3d/Config.in
index 8c1877fbc1..ba6eaa55d2 100644
--- a/package/mesa3d/Config.in
+++ b/package/mesa3d/Config.in
@@ -51,6 +51,15 @@ comment "llvm support needs a toolchain not affected by GCC bug 64735"
 	depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
 	depends on BR2_TOOLCHAIN_HAS_GCC_BUG_64735
 
+#clang and libclc dependencies are satisfied by BR2_PACKAGE_MESA3D_LLVM
+config BR2_PACKAGE_MESA3D_OPENCL
+	bool "OpenCL support"
+	depends on BR2_PACKAGE_MESA3D_LLVM
+	depends on BR2_PACKAGE_MESA3D_GALLIUM_DRIVER_R600 || \
+		BR2_PACKAGE_MESA3D_GALLIUM_DRIVER_RADEONSI
+	select BR2_PACKAGE_CLANG
+	select BR2_PACKAGE_LIBCLC
+
 # inform the .mk file of gallium, dri or vulkan driver selection
 config BR2_PACKAGE_MESA3D_GALLIUM_DRIVER
 	bool
diff --git a/package/mesa3d/mesa3d.mk b/package/mesa3d/mesa3d.mk
index fb9189741f..95ea0581df 100644
--- a/package/mesa3d/mesa3d.mk
+++ b/package/mesa3d/mesa3d.mk
@@ -237,4 +237,13 @@ else
 MESA3D_CONF_OPTS += --disable-lmsensors
 endif
 
+ifeq ($(BR2_PACKAGE_MESA3D_OPENCL),y)
+MESA3D_DEPENDENCIES += clang libclc
+MESA3D_CONF_OPTS += --enable-opencl \
+	--disable-opencl-icd \
+	--with-clang-libdir=$(STAGING_DIR)/usr/lib
+else
+MESA3D_CONF_OPTS += --disable-opencl
+endif
+
 $(eval $(autotools-package))
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [Buildroot] [PATCH v7 3/3] package/clinfo: new package
  2018-05-04 11:46 [Buildroot] [PATCH v7 0/3] Enable OpenCL support in Mesa 3D Valentin Korenblit
  2018-05-04 11:46 ` [Buildroot] [PATCH v7 1/3] package/libclc: new package Valentin Korenblit
  2018-05-04 11:46 ` [Buildroot] [PATCH v7 2/3] package/mesa3d: enable OpenCL support Valentin Korenblit
@ 2018-05-04 11:46 ` Valentin Korenblit
  2 siblings, 0 replies; 6+ messages in thread
From: Valentin Korenblit @ 2018-05-04 11:46 UTC (permalink / raw
  To: buildroot

This tool allows to verify if the OpenCL environment is set up correctly and
output information related to the supported OpenCL platforms.

Signed-off-by: Valentin Korenblit <valentin.korenblit@smile.fr>
---
 DEVELOPERS                 |  1 +
 package/Config.in          |  1 +
 package/clinfo/Config.in   |  9 +++++++++
 package/clinfo/clinfo.hash |  2 ++
 package/clinfo/clinfo.mk   | 23 +++++++++++++++++++++++
 5 files changed, 36 insertions(+)
 create mode 100644 package/clinfo/Config.in
 create mode 100644 package/clinfo/clinfo.hash
 create mode 100644 package/clinfo/clinfo.mk

diff --git a/DEVELOPERS b/DEVELOPERS
index 3f5775ff48..ccc8a847d6 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -1930,6 +1930,7 @@ F:	package/tstools/
 
 N:	Valentin Korenblit <valentin.korenblit@smile.fr>
 F:	package/clang/
+F:	package/clinfo/
 F:	package/libclc/
 F:	package/llvm/
 
diff --git a/package/Config.in b/package/Config.in
index dc0ee8a965..ece74ab9e4 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1494,6 +1494,7 @@ menu "Other"
 	source "package/clang/Config.in"
 	source "package/clapack/Config.in"
 	source "package/classpath/Config.in"
+	source "package/clinfo/Config.in"
 	source "package/cppcms/Config.in"
 	source "package/cracklib/Config.in"
 	source "package/dawgdic/Config.in"
diff --git a/package/clinfo/Config.in b/package/clinfo/Config.in
new file mode 100644
index 0000000000..279b5acbfb
--- /dev/null
+++ b/package/clinfo/Config.in
@@ -0,0 +1,9 @@
+config BR2_PACKAGE_CLINFO
+	bool "clinfo"
+	depends on BR2_PACKAGE_MESA3D_OPENCL
+	help
+	  clinfo is a simple command-line application that enumerates
+	  all possible (known) properties of the OpenCL platform and
+	  devices available on the system.
+
+	  https://github.com/Oblomov/clinfo
diff --git a/package/clinfo/clinfo.hash b/package/clinfo/clinfo.hash
new file mode 100644
index 0000000000..e1c10afc9e
--- /dev/null
+++ b/package/clinfo/clinfo.hash
@@ -0,0 +1,2 @@
+# locally calculated
+sha256 64b02e68ccff3b95437bd0bd70dcb88438c58adec16a7145a5d4e5c26a898ccf clinfo-2.2.18.03.26.tar.gz
diff --git a/package/clinfo/clinfo.mk b/package/clinfo/clinfo.mk
new file mode 100644
index 0000000000..1477b76946
--- /dev/null
+++ b/package/clinfo/clinfo.mk
@@ -0,0 +1,23 @@
+################################################################################
+#
+# clinfo
+#
+################################################################################
+
+CLINFO_VERSION = 2.2.18.03.26
+CLINFO_SITE = $(call github,Oblomov,clinfo,$(CLINFO_VERSION))
+CLINFO_LICENSE = CC0-1.0
+CLINFO_LICENSE_FILES = legalcode.txt
+
+# libOpenCL is needed
+CLINFO_DEPENDENCIES = mesa3d
+
+define CLINFO_BUILD_CMDS
+	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D) $(TARGET_CONFIGURE_OPTS)
+endef
+
+define CLINFO_INSTALL_TARGET_CMDS
+	$(INSTALL) -D -m 755 $(@D)/clinfo $(TARGET_DIR)/usr/bin/clinfo
+endef
+
+$(eval $(generic-package))
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [Buildroot] [PATCH v7 1/3] package/libclc: new package
  2018-05-04 11:46 ` [Buildroot] [PATCH v7 1/3] package/libclc: new package Valentin Korenblit
@ 2018-05-28 20:33   ` Thomas Petazzoni
  2018-05-29 15:27     ` Valentin Korenblit
  0 siblings, 1 reply; 6+ messages in thread
From: Thomas Petazzoni @ 2018-05-28 20:33 UTC (permalink / raw
  To: buildroot

Hello Valentin,

On Fri,  4 May 2018 13:46:01 +0200, Valentin Korenblit wrote:

> Given that clc headers are being installed to a non-standard location, it is
> necessary to specify this path in Mesa's configure.ac. Otherwise, pkg-config
> will output the absolute path to these headers located in STAGING_DIR, which
> will cause a runtime error when calling clBuildProgram.

It's not clear to me why the mesa3d patch needs to be within this
commit. Why isn't it introduced as part of PATCH 2/3, when you enable
libclc support in mesa3d ?

If mesa3d has a build issue because it misdetects libclc as soon as it
is available, then perhaps an explicit --disable-opencl in mesa3d.mk
added as a preliminary patch would avoid the problem.

> diff --git a/package/libclc/libclc.hash b/package/libclc/libclc.hash
> new file mode 100644
> index 0000000000..c5bf5a668c
> --- /dev/null
> +++ b/package/libclc/libclc.hash
> @@ -0,0 +1,3 @@
> +# locally calculated
> +sha256 e2f1f294f93695f2d1f87ccb9760231b435702dd0c27abeb38baa97186674d9e  libclc-00236279a293b3737dee08c14f25923a889d2795.tar.gz

Are you sure about this hash ? I don't get the same hash here:

ERROR: libclc-00236279a293b3737dee08c14f25923a889d2795.tar.gz has wrong sha256 hash:
ERROR: expected: e2f1f294f93695f2d1f87ccb9760231b435702dd0c27abeb38baa97186674d9e
ERROR: got     : 00a8de040b33e68ab7eb9cc94cb7f8a16aa56ce2ae0dadd03decc7f51eb4f674

> +# C++ compiler is used to build a small tool (prepare-builtins) for the host.
> +# It must be built with the C++ compiler from the host
> +LIBCLC_CONF_OPTS = --with-llvm-config=$(HOST_DIR)/usr/bin/llvm-config \
> +	--prefix="/usr" \
> +	--includedir="/usr/share" \
> +	--pkgconfigdir="/usr/lib/pkgconfig" \

Nit: double quotes are probably not needed around those arguments.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Buildroot] [PATCH v7 1/3] package/libclc: new package
  2018-05-28 20:33   ` Thomas Petazzoni
@ 2018-05-29 15:27     ` Valentin Korenblit
  0 siblings, 0 replies; 6+ messages in thread
From: Valentin Korenblit @ 2018-05-29 15:27 UTC (permalink / raw
  To: buildroot

Hi Thomas,

On 28/05/2018 22:33, Thomas Petazzoni wrote:
> Hello Valentin,
>
> On Fri,  4 May 2018 13:46:01 +0200, Valentin Korenblit wrote:
>> Given that clc headers are being installed to a non-standard location, it is
>> necessary to specify this path in Mesa's configure.ac. Otherwise, pkg-config
>> will output the absolute path to these headers located in STAGING_DIR, which
>> will cause a runtime error when calling clBuildProgram.
> It's not clear to me why the mesa3d patch needs to be within this
> commit. Why isn't it introduced as part of PATCH 2/3, when you enable
> libclc support in mesa3d ?

You're right, I'll move that to the next patch

> If mesa3d has a build issue because it misdetects libclc as soon as it
> is available, then perhaps an explicit --disable-opencl in mesa3d.mk
> added as a preliminary patch would avoid the problem.

Perfect, I haven't thought about that.

>> diff --git a/package/libclc/libclc.hash b/package/libclc/libclc.hash
>> new file mode 100644
>> index 0000000000..c5bf5a668c
>> --- /dev/null
>> +++ b/package/libclc/libclc.hash
>> @@ -0,0 +1,3 @@
>> +# locally calculated
>> +sha256 e2f1f294f93695f2d1f87ccb9760231b435702dd0c27abeb38baa97186674d9e  libclc-00236279a293b3737dee08c14f25923a889d2795.tar.gz
> Are you sure about this hash ? I don't get the same hash here:
>
> ERROR: libclc-00236279a293b3737dee08c14f25923a889d2795.tar.gz has wrong sha256 hash:
> ERROR: expected: e2f1f294f93695f2d1f87ccb9760231b435702dd0c27abeb38baa97186674d9e
> ERROR: got     : 00a8de040b33e68ab7eb9cc94cb7f8a16aa56ce2ae0dadd03decc7f51eb4f674

I've just updated the package to a more recent commit, also its hash.

>> +# C++ compiler is used to build a small tool (prepare-builtins) for the host.
>> +# It must be built with the C++ compiler from the host
>> +LIBCLC_CONF_OPTS = --with-llvm-config=$(HOST_DIR)/usr/bin/llvm-config \
>> +	--prefix="/usr" \
>> +	--includedir="/usr/share" \
>> +	--pkgconfigdir="/usr/lib/pkgconfig" \
> Nit: double quotes are probably not needed around those arguments.

I'll remove them.

> Thomas

Best regards and thanks for the review,

Valentin

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2018-05-29 15:27 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-05-04 11:46 [Buildroot] [PATCH v7 0/3] Enable OpenCL support in Mesa 3D Valentin Korenblit
2018-05-04 11:46 ` [Buildroot] [PATCH v7 1/3] package/libclc: new package Valentin Korenblit
2018-05-28 20:33   ` Thomas Petazzoni
2018-05-29 15:27     ` Valentin Korenblit
2018-05-04 11:46 ` [Buildroot] [PATCH v7 2/3] package/mesa3d: enable OpenCL support Valentin Korenblit
2018-05-04 11:46 ` [Buildroot] [PATCH v7 3/3] package/clinfo: new package Valentin Korenblit

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.