[Makefile] Drop SGX_RUN

For detection of SGX/non-SGX (for example in regression tests) always
use the SGX environment variable. To generate launch/EINIT tokens use
the new 'sgx-tokens' Make target.

CONTRIBUTING.md

@@ -109,8 +109,7 @@ For SGX, one needs to do the following:
 
 ```Bash
 cd Pal/regression
-make SGX=1
-make SGX_RUN=1 regression
+make SGX=1 regression
 ```
 
 If a test fails unexpectedly, one can use the KEEP_LOG=1 option to get the complete output.

Documentation/manpages/pal_loader.rst

@@ -34,9 +34,4 @@ Environment variables
 .. envvar:: SGX
 
    If not empty and not ``0``, enable :term:`SGX`. Could be used instead of
-   :option:`SGX option <SGX>`. This has some unexplained interaction with
-   :envvar:`SGX_RUN`.
-
-.. envvar:: SGX_RUN
-
-   This is a mystery to me. It cannot be set together with :envvar:`SGX`.
+   :option:`SGX option <SGX>`.

Documentation/oldwiki/Debugging-Graphene.md

@@ -33,7 +33,7 @@ For instance, if you want to debug the `helloworld` program, run the following c
 
     cd LibOS/shim/test/native
     make SGX=1
-    make SGX_RUN=1
+    make SGX=1 sgx-tokens
 
 To run Graphene with GDB, use the Graphene loader (`pal_loader`) and specify `GDB=1`:
 

Documentation/oldwiki/Graphene-SGX-Quick-Start.md

@@ -46,7 +46,7 @@ The first command should list `isgx` and the second command should list the proc
 
     cd $GRAPHENE_DIR/LibOS/shim/test/native
     make SGX=1
-    make SGX_RUN=1
+    make SGX=1 sgx-tokens
     SGX=1 ./pal_loader helloworld
 
 ### 8. Test LMBench Application

Documentation/oldwiki/Introduction-to-Graphene-SGX.md

@@ -101,8 +101,8 @@ token tool `Pal/src/host/Linux-SGX/signer/pal-sgx-get-token` to connect to the a
 and retrieve the token.
 
 For applications that are prepared in the Graphene apps directory (GCC, Apache, Bash, etc.), type
-`make SGX_RUN=1` in the corresponding directory. The scripts are automated to retrieve the tokens
-for the applications.
+`make SGX=1 sgx-tokens` in the corresponding directory. The scripts are automated to retrieve the
+tokens for the applications.
 
 With the manifest (`.manifest.sgx`), the signature (`.sig`), and the token (`.token`) ready, one
 can launch Graphene-SGX to run the application. Graphene-SGX provides three options for specifying

Documentation/oldwiki/Introduction-to-Graphene.md

@@ -193,7 +193,7 @@ For more details about running tested/benchmarked applications in Graphene, plea
 
 - Generate launch tokens from the aesmd service:
 
-      make SGX_RUN=1
+      make SGX=1 sgx-tokens
 
 - Run `helloworld` with Graphene-SGX:
 
@@ -207,7 +207,7 @@ For more details about running tested/benchmarked applications in Graphene, plea
 
 - Generate a launch token from the aesmd service:
 
-      make SGX_RUN=1
+      make SGX=1 sgx-tokens
 
 - Run the `helloworld.py` script with Graphene-SGX:
 

Documentation/oldwiki/Run-Applications-in-Graphene-SGX.md

@@ -18,7 +18,7 @@ and run LMBench in a native environment and under Graphene-SGX:
 
     cd LibOS/shim/test/apps/lmbench
     make SGX=1                    # compile lmbench and generate manifest and signature
-    make SGX_RUN=1                # get enclave token
+    make SGX=1 sgx-tokens         # get enclave token
     make SGX=1 test-graphene      # run the whole package in Graphene-SGX
 
 The result of native runs can be found in `lmbench-2.5/results/linux`. The result of Graphene-SGX
@@ -42,7 +42,7 @@ To run Python, first generate the manifest and the signature, and retrieve the t
 
     cd LibOS/shim/test/apps/python
     make SGX=1
-    make SGX_RUN=1
+    make SGX=1 sgx-tokens
 
 You can run `python.manifest.sgx` as an executable to load any script. The manifest file is
 actually a script with a shebang that can be automatically loaded in PAL. Use the following
@@ -57,7 +57,7 @@ To run R, first prepare the manifest:
 
     cd LibOS/shim/test/apps/r
     make SGX=1
-    make SGX_RUN=1
+    make SGX=1 sgx-tokens
 
 You can run `R.manifest.sgx` as an executable to load any script. The manifest file is actually
 a script with a shebang that can be automatically loaded in PAL. Use the following commands:
@@ -72,7 +72,7 @@ in `LibOS/shim/test/apps/lighttpd`. To build Lighttpd, run the following command
 
     cd LibOS/shim/test/apps/lighttpd
     make SGX=1
-    make SGX_RUN=1
+    make SGX=1 sgx-tokens
 
 The commands above will compile the source code, build the manifest file for Graphene-SGX, generate
 the configuration file for Lighttpd, and generate the HTML sample files. We prepared the following file
@@ -107,7 +107,7 @@ Apache, run the following commands:
 
     cd LibOS/shim/test/apps/apache
     make SGX=1
-    make SGX_RUN=1
+    make SGX=1 sgx-tokens
 
 The commands above will compile the source code, build the manifest file for Graphene, generate
 the configuration file for Apache, and generate the HTML sample files (same as described in the
@@ -133,7 +133,7 @@ the proper manifest, run the following commands:
 
     cd LibOS/shim/test/apps/busybox
     make SGX=1
-    make SGX_RUN=1
+    make SGX=1 sgx-tokens
 
 To run Busybox, you may directly run busybox.manifest built in the directory as a script.
 For example:
@@ -152,7 +152,7 @@ the following commands:
 
     cd LibOS/shim/test/apps/bash
     make SGX=1
-    make SGX_RUN=1
+    make SGX=1 sgx-tokens
 
 To test Bash, use the benchmark suites we prepared: `bash_test.sh` and `unixbench`. Run one of the
 following commands to test Bash:

Jenkinsfiles/Linux-SGX

@@ -30,7 +30,7 @@ pipeline {
                             make -j 8 SGX=1 WERROR=1 test
                         '''
                         sh '''
-                            make SGX_RUN=1 test
+                            make SGX=1 sgx-tokens
                         '''
                     }
                 }
@@ -46,55 +46,49 @@ pipeline {
                                 else \
                                     make SGX=1; \
                                 fi
-                                make SGX_RUN=1
-                                make SGX_RUN=1 KEEP_LOG=1 regression
+                                make SGX=1 all sgx-tokens
+                                make SGX=1 KEEP_LOG=1 regression
                                 '''
                         }
                         timeout(time: 15, unit: 'MINUTES') {
                             sh '''
                                 cd LibOS/shim/test/regression
-                                make SGX=1
-                                make SGX_RUN=1
-                                make SGX_RUN=1 regression
+                                make SGX=1 all sgx-tokens
+                                make SGX=1 regression
                             '''
                         }
                         timeout(time: 5, unit: 'MINUTES') {
                             sh '''
                                 cd LibOS/shim/test/apps/python
-                                make SGX=1
-                                make SGX_RUN=1
-                                make SGX_RUN=1 regression
+                                make SGX=1 all sgx-tokens
+                                make SGX=1 regression
                             '''
                         }
                         timeout(time: 5, unit: 'MINUTES') {
                             sh '''
                                 cd LibOS/shim/test/apps/bash
-                                make SGX=1
-                                make SGX_RUN=1
-                                make SGX_RUN=1 regression
+                                make SGX=1 all sgx-tokens
+                                make SGX=1 regression
                            '''
                         }
                         timeout(time: 10, unit: 'MINUTES') {
                             sh '''
                                 cd LibOS/shim/test/apps/gcc
-                                make SGX=1
-                                make SGX_RUN=1
-                                make SGX_RUN=1 regression
+                                make SGX=1 all sgx-tokens
+                                make SGX=1 regression
                            '''
                         }
                         timeout(time: 15, unit: 'MINUTES') {
                             sh '''
                                 cd LibOS/shim/test/apps/lmbench
-                                make SGX=1
-                                make SGX_RUN=1
-                                make SGX_RUN=1 regression N_RUNS=1 ENOUGH=100
+                                make SGX=1 all sgx-tokens
+                                make SGX=1 regression N_RUNS=1 ENOUGH=100
                             '''
                         }
                         sh '''
                             cd LibOS/shim/test/apps/lighttpd
-                            make SGX=1
-                            make SGX_RUN=1
-                            make SGX_RUN=1 start-graphene-server &
+                            make SGX=1 all sgx-tokens
+                            make SGX=1 start-graphene-server &
                             sleep 10
                             LOOP=1 CONCURRENCY_LIST="1 32" ./benchmark-http.sh 127.0.0.1:8000
                             '''

Jenkinsfiles/Linux-SGX-18.04

@@ -27,7 +27,7 @@ pipeline {
                             make -j 8 SGX=1 WERROR=1 test
                         '''
                         sh '''
-                            make SGX_RUN=1 test
+                            make SGX=1 sgx-tokens
                         '''
                     }
                 }
@@ -43,55 +43,49 @@ pipeline {
                                 else \
                                     make SGX=1; \
                                 fi
-                                make SGX_RUN=1
-                                make SGX_RUN=1 KEEP_LOG=1 regression
+                                make SGX=1 all sgx-tokens
+                                make SGX=1 KEEP_LOG=1 regression
                                 '''
                         }
                         timeout(time: 15, unit: 'MINUTES') {
                             sh '''
                                 cd LibOS/shim/test/regression
-                                make SGX=1
-                                make SGX_RUN=1
-                                make SGX_RUN=1 regression
+                                make SGX=1 all sgx-tokens
+                                make SGX=1 regression
                             '''
                         }
                         timeout(time: 5, unit: 'MINUTES') {
                             sh '''
                                 cd LibOS/shim/test/apps/python
-                                make SGX=1
-                                make SGX_RUN=1
-                                make SGX_RUN=1 regression
+                                make SGX=1 all sgx-tokens
+                                make SGX=1 regression
                             '''
       }
       timeout(time: 5, unit: 'MINUTES') {
                             sh '''
                                 cd LibOS/shim/test/apps/bash
-                                make SGX=1
-                                make SGX_RUN=1
-                                make SGX_RUN=1 regression
+                                make SGX=1 all sgx-tokens
+                                make SGX=1 regression
                            '''
       }
                         timeout(time: 10, unit: 'MINUTES') {
                             sh '''
                                 cd LibOS/shim/test/apps/gcc
-                                make SGX=1
-                                make SGX_RUN=1
-                                make SGX_RUN=1 regression
+                                make SGX=1 all sgx-tokens
+                                make SGX=1 regression
                            '''
                         }
                         timeout(time: 15, unit: 'MINUTES') {
                             sh '''
                                 cd LibOS/shim/test/apps/lmbench
-                                make SGX=1
-                                make SGX_RUN=1
-                                make SGX_RUN=1 regression N_RUNS=1 ENOUGH=100
+                                make SGX=1 all sgx-tokens
+                                make SGX=1 regression N_RUNS=1 ENOUGH=100
                             '''
                         }
                         sh '''
                             cd LibOS/shim/test/apps/lighttpd
-                            make SGX=1
-                            make SGX_RUN=1
-                            make SGX_RUN=1 start-graphene-server &
+                            make SGX=1 all sgx-tokens
+                            make SGX=1 start-graphene-server &
                             sleep 10
                             LOOP=1 CONCURRENCY_LIST="1 32" ./benchmark-http.sh 127.0.0.1:8000
                             '''

LibOS/Makefile

@@ -143,3 +143,7 @@ endif
 .PHONY: test
 test:
 	$(MAKE) -C $(SHIM_DIR) test
+
+.PHONY: sgx-tokens
+sgx-tokens:
+	$(MAKE) -C $(SHIM_DIR) sgx-tokens

LibOS/shim/Makefile

@@ -11,6 +11,10 @@ all:
 test:
 	$(MAKE) -C test
 
+.PHONY: sgx-tokens
+sgx-tokens:
+	$(MAKE) -C test sgx-tokens
+
 .PHONY: clean
 clean:
 	$(MAKE) -C src clean

LibOS/shim/test/Makefile

@@ -7,8 +7,8 @@ export SYS
 .PHONY: default
 default: all
 
-.PHONY: all clean
-all clean:
+.PHONY: all clean sgx-tokens
+all clean sgx-tokens:
 	for d in $(SUBDIRS); do \
 		$(MAKE) -C $$d $@; \
 	done

LibOS/shim/test/Makefile.Test

@@ -35,6 +35,9 @@ include $(PALDIR)/Makefile.Test
 .PHONY: all
 all: pal_loader $(target) | $(call expand_target,$(exec_target))
 
+.PHONY: sgx-tokens
+sgx-tokens: $(call expand_target_to_token,$(exec_target))
+
 ifeq ($(DEBUG),1)
 CC += -g
 CXX += -g

LibOS/shim/test/apps

@@ -1 +1 @@
-Subproject commit b3e6679fab7e06aed518aa49b41d3e6edf85ddec
+Subproject commit 60d0dff702235142ed61fb48164985736e01c634

Makefile

@@ -1,7 +1,13 @@
 SYS ?= $(shell gcc -dumpmachine)
 export SYS
 
-targets = all clean format test
+targets = all clean format test sgx-tokens
+
+ifneq ($(filter sgx-tokens,$(MAKECMDGOALS)),)
+ifneq ($(SGX),1)
+$(error "The 'sgx-tokens' target requires SGX=1")
+endif
+endif
 
 .PHONY: $(targets)
 $(targets):

Pal/Makefile

@@ -18,6 +18,11 @@ test:
 	$(MAKE) -C test
 	$(MAKE) -C regression
 
+.PHONY: sgx-tokens
+sgx-tokens:
+	$(MAKE) -C test sgx-tokens
+	$(MAKE) -C regression sgx-tokens
+
 .PHONY: format
 format:
 	clang-format -i $(shell find . -path ./lib/crypto/mbedtls -prune -o \

Pal/regression/Makefile

@@ -26,6 +26,9 @@ export PAL_LOADER = $(RUNTIME_DIR)/pal-$(PAL_HOST)
 .PHONY: all
 all: $(call expand_target,$(target)) $(preloads)
 
+.PHONY: sgx-tokens
+sgx-tokens: $(call expand_target_to_token,$(target))
+
 ifeq ($(DEBUG),1)
 CC += -g
 endif
@@ -104,7 +107,7 @@ regression:
 	$(RM) pal-regression.xml
 	$(MAKE) pal-regression.xml
 
-pal-regression.xml: test_pal.py $(call expand_target,$(target))
+pal-regression.xml: test_pal.py $(call expand_target_to_token,$(target))
 	python3 -m pytest --junit-xml $@ -v test_pal.py
 
 .PHONY: clean

Pal/src/Makefile

@@ -159,3 +159,7 @@ clean:
 .PHONY: test
 test:
 	$(MAKE) -C $(HOST_DIR) test
+
+.PHONY: sgx-tokens
+sgx-tokens:
+	$(MAKE) -C $(HOST_DIR) sgx-tokens

Pal/src/Makefile.Host

@@ -11,11 +11,11 @@ $(error Unsupported platform: $(SYS))
 endif
 endif
 
-# Set SGX=1 to build Graphene for SGX
-ifeq ($(SGX)$(SGX_RUN),11)
-	$(error "Do not use SGX=1 and SGX_RUN=1 at the same time")
+ifeq ($(SGX_RUN),1)
+	$(error "SGX_RUN has been removed. Always set SGX=1 if building for SGX and use the 'sgx-tokens' make target to build launch/EINIT tokens")
 endif
-ifeq ($(SGX)$(SGX_RUN),1)
+
+ifeq ($(SGX),1)
 	PAL_HOST := $(patsubst %-SGX,%,$(PAL_HOST))-SGX
 endif
 

Pal/src/host/Linux-SGX/Makefile.Test

@@ -6,15 +6,11 @@ SGX_SIGNER_KEY ?= $(SGX_DIR)/signer/enclave-key.pem
 SGX_SIGN = $(SGX_DIR)/signer/pal-sgx-sign -libpal $(LIBPAL) -key $(SGX_SIGNER_KEY)
 SGX_GET_TOKEN = $(SGX_DIR)/signer/pal-sgx-get-token
 
-ifeq ($(SGX_RUN),1)
-
-expand_target = $(foreach t,$(filter-out manifest,$(1)),$(patsubst %.manifest,%,$(t)).token)
+expand_target_to_token = $(foreach t,$(filter-out manifest,$(1)),$(patsubst %.manifest,%,$(t)).token)
 
 %.token: %.sig
 	$(call cmd,sgx_get_token)
 
-else
-
 expand_target = $(1) $(foreach t,$(filter-out manifest,$(1)), \
 		$(patsubst %.manifest,%,$(t)).manifest.sgx)
 
@@ -44,7 +40,3 @@ $(SGX_SIGNER_KEY):
 # use static pattern rules in the corresponding Makefiles (e.g., in Pal/regression).
 # %.manifest.sgx.d: %.manifest
 # 	$(call cmd,sgx_sign_depend)
-
-*.token nothing:
-
-endif

Pal/src/host/Linux-SGX/sgx_main.c

@@ -887,7 +887,7 @@ static int load_enclave (struct pal_enclave * enclave,
     if (IS_ERR(enclave->token)) {
         SGX_DBG(DBG_E, "Cannot open token \'%s\'. Use \'"
                 PAL_FILE("pal-sgx-get-token")
-                "\' on the runtime host or run \'make SGX_RUN=1\' "
+                "\' on the runtime host or run \'make SGX=1 sgx-tokens\' "
                 "in the Graphene source to create the token file.\n",
                 token_uri);
         free(token_uri);

Pal/test/Makefile

@@ -23,6 +23,9 @@ pal_lib = ../../Runtime/libpal-$(PAL_HOST).so
 .PHONY: all
 all:	pal_loader $(call expand_target,$(target))
 
+.PHONY: sgx-tokens
+sgx-tokens: $(call expand_target_to_token,$(target))
+
 ifeq ($(DEBUG),1)
 CC += -g
 endif

README.rst

@@ -156,7 +156,7 @@ a |_| few tested applications, such as GCC, Python, and Apache.
 
    - Generate the token from aesmd service, via command::
 
-      make SGX_RUN=1
+      make SGX=1 sgx-tokens
 
    - Run Hello World program with Graphene on SGX::
 
@@ -174,7 +174,7 @@ a |_| few tested applications, such as GCC, Python, and Apache.
 
    - Generate token::
 
-      make SGX_RUN=1
+      make SGX=1 sgx-tokens
 
    - Run python helloworld with Graphene-SGX via::
 
@@ -210,7 +210,7 @@ To enable remote attestation tests in ``Pal/regression``, specify the following
 
     cd PAL/regression
     make SGX=1 RA_CLIENT_SPID=<SPID> RA_CLIENT_KEY=<KEY>
-    make SGX_RUN=1
+    make SGX=1 sgx-tokens
 
 
 If you receive a "GROUP_OUT_OF_DATE" status from IAS, this status indicates that your CPU

Runtime/Makefile

@@ -10,3 +10,6 @@ format:
 
 .PHONY: test
 test:
+
+.PHONY: sgx-tokens
+sgx-tokens:

Runtime/pal_loader

@@ -16,18 +16,6 @@ do
 	shift
 done
 
-if [ -n "$SGX" ] && [ "$SGX" != "0" ]; then
-	export SGX=1
-	# Sometimes, we end up with a stray SGX_RUN in the environment,
-	# which makes the Makefile.Host unhappy
-	unset SGX_RUN
-	# The interaction of SGX and SGX_RUN is getting pretty unwieldly.
-	# We should kill off SGX_RUN.  Here, we can get in trouble
-	# if the make invocation below gets SGX_RUN via an MAKEFLAGS
-	# from a wrapper makefile (e.g., the regression tests)
-	unset MAKEFLAGS
-fi
-
 RUNTIME_DIR=$(/usr/bin/dirname $(readlink -f ${BASH_SOURCE[0]}))
 PAL_HOST=$(/usr/bin/make --no-print-directory --quiet -f $RUNTIME_DIR/../Pal/src/Makefile.Host print_host 2>&1)
 

Scripts/Makefile

@@ -10,3 +10,6 @@ format: ;
 
 .PHONY: test
 test: ;
+
+.PHONY: sgx-tokens
+sgx-tokens: ;

Scripts/regression.py

@@ -5,7 +5,7 @@ import signal
 import subprocess
 import unittest
 
-HAS_SGX = os.environ.get('SGX_RUN') == '1'
+HAS_SGX = os.environ.get('SGX') == '1'
 
 def expectedFailureIf(predicate):
     if predicate: