Wrong tasks routing - DEFAULT_VERSION_HOSTNAME env variable set to empty string #118

Activity

1. duggelz commented on Jun 29, 2017

   

   duggelz

   on Jun 29, 2017

   Contributor

   More actions

   Thanks for filing this issue.

   Can you tell me whether your app.yaml file has vm: true or env: flex? The first indicates that you are using the Managed VM environment, the second indicates that you're using the App Engine Flexible environment. The names and contents of many environment variables changed between those two environments, and it sounds like the code may not handle both.

   Please note that the beta Managed VMs environment is deprecated in favor of the App Engine Flexible environment, which is GA. And this beta python-compat runtime is deprecated in favor of the python runtime, which is also GA. See Managed VMs upgrade guide for more information.
2. ntassev commented on Jun 29, 2017

   

   ntassev

   on Jun 29, 2017

   Author

   More actions

   It is env:flex - here is the excerpt from the descriptor some-service.yaml (it is not the default, hence not app.yaml. The default module and other services are not on VMs but on GAE classic):

   service: some-service
   runtime: python-compat
   env: flex
   entrypoint: gunicorn -b :$PORT main:app
   runtime_config:
     python_version: 2
   beta_settings:
     enable_app_engine_apis: true
   

   It used to be vm:true, but we migrated to take advantage of the extra time while python-compat will be available (until October). We have code that depends on NDB and taskqueues, so the python Flex runtime is not an option right now.

   I checked the env variables available both in the container and during processing an HTTP request and DEFAULT_VERSION_HOSTNAME is always an empty string.
3. ntassev commented on Jun 29, 2017

   

   ntassev

   on Jun 29, 2017

   Author

   More actions

   Speaking more broadly on the topic, we're using the following shims, where we discovered some fluctuations in behavior between the four runtimes - GAE classic, Managed VM vm:true, Flex with python-compat and Flex with python I'm not 100% sure they are completely accurate, but I'd be glad if you or somebody comments on them or if they'd help somebody facing migration issues.

   def is_gae_classic():
       return ('GCLOUD_PROJECT' not in os.environ) and ('GAE_VM' not in os.environ)
   
   
   def is_flex_env():
       return 'GCLOUD_PROJECT' in os.environ
   
   
   def get_current_module_name():
       try:
           # works only on GAE proper
           from google.appengine.api import modules
           name = modules.get_current_module_name()
           if not name:
               raise Exception()
           return name
       except:
           # for `vm:true` and `env:flex python-compat`
           if 'GAE_MODULE_NAME' in os.environ:
               return os.environ.get('GAE_MODULE_NAME')
           else:
               # on new `env:flex`
               return os.environ['GAE_SERVICE']
   
   
   def get_application_id():
       try:
           # works on: GAE proper, `vm:true`, `env:flex python-compat`
           from google.appengine.api import app_identity
           return app_identity.get_application_id()
       except:
           if 'GAE_LONG_APP_ID' in os.environ:
               return os.environ['GAE_LONG_APP_ID']
           else:
               # on new `env:flex`
               return os.environ.get('GCLOUD_PROJECT')
   
   
   def get_current_version_name():
       try:
           # works on: GAE proper
           from google.appengine.api import modules
           name = modules.get_current_version_name()
           if not name:
               raise Exception()
           return name
       except:
           if 'GAE_MODULE_VERSION' in os.environ:
               # on `vm:true`
               return os.environ['GAE_MODULE_VERSION']
           else:
               # on new `env:flex` and `env:flex python-compat`
               return os.environ['GAE_VERSION']
   
   
   def _get_current_version_timestamp_string():
       if 'CURRENT_VERSION_ID' in os.environ:
           # on: GAE proper
           return os.environ['CURRENT_VERSION_ID'].split('.')[-1]
       elif 'GAE_DEPLOYMENT_ID' in os.environ:
           # on `vm:true`, `env:flex` and `env:flex python-compat`
           return os.environ['GAE_DEPLOYMENT_ID']
   
   
   def get_current_version_id():
       if 'CURRENT_VERSION_ID' in os.environ:
           # on: GAE proper
           return os.environ['CURRENT_VERSION_ID']
       else:
           get_current_version_name() + '.' + _get_current_version_timestamp_string()
   
   
   def get_default_version_hostname():
       try:
           # on: GAE proper
           from google.appengine.api import app_identity
           hostname = app_identity.get_default_version_hostname()
           if not hostname:
               raise Exception()
           return hostname
       except:
           if 'GAE_APPENGINE_HOSTNAME' in os.environ:
               # on `vm:true` and `env:flex python-compat`
               return os.environ['GAE_APPENGINE_HOSTNAME']
           else:
              # not available on new `env:flex`, so DIY
              return '{}.appspot.com'.format(get_application_id())
   
4. duggelz commented on Jun 29, 2017

   

   duggelz

   on Jun 29, 2017

   Contributor

   More actions

   CC @jonparrott @liyanhui1228
5. duggelz commented on Jun 29, 2017

   

   duggelz

   on Jun 29, 2017

   Contributor

   More actions

   We continue to have discussions about how best to detect/communicate the current environment (GCE, GKE, various flavors of GAE, etc) to application code. However, I can point you to the logic our Cloud client libraries use today:

   https://github.com/GoogleCloudPlatform/google-cloud-python/blob/master/logging/google/cloud/logging/client.py#L52

   And the logic used by the auth client:

   https://github.com/GoogleCloudPlatform/google-auth-library-python/blob/master/google/auth/_default.py#L192
6. duggelz commented on Jun 29, 2017

   

   duggelz

   on Jun 29, 2017

   Contributor

   More actions

   CC @JustinBeckwith