Solvedcloudml samples No instance key in output of canned estimator census sample

As other individuals have noted:

According to the ML Engine documentation, an instance key is required to match the returned predictions with the input data (Source)

Would it be possible to modify the canned estimator census sample to include the instance key in the output?

This was addressed on StackOverflow but it's not clear to me how to make this modification.

Thank you in advance for your help!

55 Answers

✔️Accepted Answer

Ah, I see. After making this modification it works. Thanks so much for your help @elibixby.

KEY = 'gender'
def key_model_fn_gen(estimator):
    def _model_fn(features, labels, mode):
        key = features.pop(KEY)
        params = estimator.params
        model_fn_ops = estimator._model_fn(features=features, labels=labels, mode=mode, params=params)
        model_fn_ops.predictions[KEY] = key
        model_fn_ops.output_alternatives[None][1][KEY] = key # <== UPDATED
        print(model_fn_ops.output_alternatives)
        return model_fn_ops
    return _model_fn

For others' reference, this is what the model_fn_ops.output_alternatives prints out as:

 {None: (3, {'probabilities': <tf.Tensor 'binary_logistic_head/predictions/probabilities:0' shape=(?, 2) dtype=float32>, 'classes': <tf.Tensor 'binary_logistic_head/_classification_output_alternatives/classes_tensor:0' shape=(?, ?) dtype=string>, 'gender': <tf.Tensor 'ExpandDims:0' shape=(?, 1) dtype=string>})} 

Other Answers:

generate_input_fn is for training or evaluation input functions (note that it reads data from files).
json_input_fn is for building a SavedModel binary (note that it uses placeholders).

Since you only need keys in prediction (i.e. for batch prediction or online prediction), you only need to add a key placeholder to the tensors expected by, and returned from the json_input_fn I'll add this context to the the StackOverflow answer

@elibixby Sounds like a great change to include.

@dwujellyfish - I ran into the same 'identity ... is both fed and fetched' issue in my json_serving_input_fn (but not my csv_serving_input_fn, since the CSV input records consist of just a single string Tensor field).

This is what my json_serving_input_fn looks like -- I had to pass separate 'inputs' and 'features' dicts to the tf.estimator.export.ServingInputReceiver call, with the tf.identity applied to the instance key in the 'features' dict, not the 'inputs' dict.

def json_serving_input_fn():
  inputs = {}
  features = {}

  for feat in INPUT_COLUMNS:
    inputs[feat.name] = tf.placeholder(shape=[None], dtype=feat.dtype)
    if feat.name == INSTANCE_KEY_COLUMN:
      features[feat.name] = tf.identity(inputs[feat.name])
    else:
      features[feat.name] = inputs[feat.name]

  # inputs: 'user_id': <tf.Tensor 'Placeholder:0' shape=(?,) dtype=int64>}
  # features: 'user_id': <tf.Tensor 'Identity:0' shape=(?,) dtype=int64>}

  serving_input_rcvr = tf.estimator.export.ServingInputReceiver(features, inputs)
  
  return serving_input_rcvr

Folks, I finally got around to writing a guide on how to do this. Hopefully, the two caveats at the end of the post explain the problems that people have been running into when using forward_features, and how to work around them:

https://medium.com/@lakshmanok/how-to-extend-a-canned-tensorflow-estimator-to-add-more-evaluation-metrics-and-to-pass-through-ddf66cd3047d

https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/courses/machine_learning/deepdive/07_structured/babyweight/trainer/model.py#L157

Related Issues:

5
cloudml samples No instance key in output of canned estimator census sample
Ah I see After making this modification it works Thanks so much for your help @elibixby For others' ...
96
sops Cannot decrypt with GPG 2.2.5 and SOPS 3.0.0
The problem suddenly re-occured.. I think it has to do with the gpg-agent For the moment this solved...
19
external dns Is it possible to use external-dns with a Traefik 2.0 IngressRoute resource?
Actually I don't think this topic should be closed I was able to use external-dns with AWS Route53 a...
15
external dns Disable external-dns for specific ingresses
This is great it works perfectly Instead of targeting specific groups from external-dns i've standar...
14
external dns AWS API InvalidClientTokenId: The security token included in the request is invalid
Hi @hickey I had faced the same error as you But later I figured out I made the mistake when putting...
13
pulumi Resource Adoption
I have a workaround until this feature is shipped While pondering #1654 ...
12
external dns External-dns fails to extract targets for SRV records on Azure
Same issue here but with NS.. time=2020-12-17T19:50:38Z level=error msg=Failed to extract targets fo...
12
external dns Ingress hosts are not getting DNS names created.
@cabrinoob - If you are using the nginx-ingress helm chart look at within the values.yaml ...
6
external dns AWS cross account access with OIDC provider in EKS & externalDNS not working
I tested it in our setup and it seems to work fine Here's what I did Let's assume we want to run Ext...
3
external dns External DNS always uses ALIAS for AWS ELBs, not CNAME (only A records are created in Route53)
It's currently not possible to force using CNAME when ALIAS would also work as our expectations was ...
3
pulumi Python Outputs to string do not work as documented
Thanks for the clarifications and feedback I agree we can definitely work toward making the document...
3
pulumi Grpc.Core.RpcException: "Failed to deserialize response message." when using "cert-manager.crds.yaml" for ConfigFile
Here is the draft of the forking plan: Fork protocolbuffers/protobuf to Pulumi org (done): https://g...
264
tqdm Jupyterlab and tqdm_notebook
I confirm this problem on my environment but it is upstream so IMHO nothing can be done on tqdm Plea...
62
CapsNet Keras pydot issue
I'm writing this here in case anyone finds this again Hi I got the following error message (copied b...
50
tensorflow onnx import frozen graph with error "Input 0 of node X was passed float from Y:0 incompatible with expected float_ref."
@OneDirection9 Here is what I ended up with Let me what you think. Note: create this issue for anybo...
47
tqdm How to update single progress bar in multiprocessing map()
from https://stackoverflow.com/questions/41920124/multiprocessing-use-tqdm-to-display-a-progress-bar...
46
keras attention mechanism Questions on implementation details
Being confused about why attention can learn info about specific index in input sequence I went on a...
44
CapsNet Keras ImportError: Failed to import pydot. Please install pydot. For example with pip install pydot.
Try this :) from keras.utils import plot_model plot_model(model) Showing this error ImportError: Fai...
31
onnx Dynamic dummy input when exporting a PyTorch model?
For now I temporarily modify the onnx graph inputs: This seems to work with the tensorflow backend b...
29
Mask_RCNN BBOX_STD_DEV usage is unclear and possibly incorrect?
@simonm3 The regressor performs best if its outputs have a mean of zero and a standard deviation of ...
26
Mask_RCNN ValueError: Error when checking input: expected input_image_meta to have shape (13,) but got array with shape (14,)
@kvigulis just in case you were still struggling.. the same dataset I create of my own doing well Bu...
14
Mask_RCNN Training cause ERROR:root:Error processing image
Just in case if anyone who's getting IndexError stumbles upon this issue list: As @zungam stated it'...
14
tqdm tqdm.pandas() ImportError: cannot import 'PanelGroupBy'
should be fixed in tqdm>=4.23.4 When using tqdm v4.23.3 and pandas v0.23.0 I get this error: It look...
13
client Error communicating with backend
Issue-Label Bot is automatically applying the label bug to this issue with a confidence of 0.67 ...
12
autokeras AttributeError: 'tuple' object has no attribute 'shape'
Hi everyone I am really sorry about the bugs I am busy fixing them and aiming at a new stable releas...
11
horovod mpi_lib.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN10tensorflow12OpDefBuilder4AttrESs
@michaelkyu @Aki57 I've been able to repro this internally and found a solution The root cause of th...
11
Mask_RCNN error: operands could not be broadcast together with shapes (1280,1280,4) (3,)
@eyildiz-ugoe My training has gone fine but I'm getting the following error when I try to infer from...
10
keras vis InvalidArgumentError: conv2d_1_input_1:0 is both fed and fetched
I manually deleted keras-vis and installed it again with ip install git+https://github.com/raghakot/...
10
tqdm Wrong output in Windows console
That's partial unicode support You can probably fix it with tqdm(... ascii=True) ...
7
Mask_RCNN How to keep model loaded while running detection in a web service?
@zungam I found the answer about an hour ago but thanks for the quick reply! :) @mtcld check out my ...
6
horovod hvd.init() hangs on aws deep learning instance
@gururao001 on AWS you should uninstall MPI that comes with their base image: Hello ...
6
horovod WARNING: One or more tensors were submitted to be reduced, gathered
I see the same thing happens pretty regularly The data is coming from network storage Horovod is run...
6
Mask_RCNN Test on own dataset / No instances to display
I've solved this problem by modifying utils.py line 866 from shift = np.array([0 0 1 1]) to shift = ...
6
client CLI: Windows with Anaconda prompt — W&B process failed to launch
For anyone hitting this error on windows Wandb version: 0.8.1 Python 3.6 Amazon DLAMI 23 environment...
5
autokeras Support python generators
@ciberger this code seems to work you can give a shot: Then you can feed your fit function with the ...
5
client Network error (ReadTimeout), entering retry loop during sweep
Issue-Label Bot is automatically applying the label bug to this issue with a confidence of 0.85 ...
4
horovod Building Horovod from source
@libliang bdist_wheel will build you a wheel file on that can be installed on a server with exactly ...
3
PINTO_model_zoo Is there an easy way to convert ONNX or PB from (NCHW) to (NHWC)?
Thank you for commenting on this for a hobbyist like me who does DeepLearning as a hobby I am not an...
3
einops [Feature suggestion] Identifiers not on both sides of the expression
@jotaf98 it is done and available in master branch Axes of size 1 can be written as 1 not () ...
34
nodejs docs samples How to add the environment variables in Windows?
Why is it looking at the wrong project? Make sure your GCLOUD_PROJECT environment variable is set to...
14
Tone.js AudioContext "not allowed to start"
a simple solution is to add this: inside your mousedown listener System: Windows 10.0.16299 Pro x64 ...
12
nodejs docs samples Unable to record speech
launch brew install sox on OSX fixed it for me I am newbie to this Google and its API's I was able t...