Hot questions for Using Azure in json

Question:

This is what i tried on Android to compare if two faces belong to the same person. I have also presented my error code at the end. Does this have something to do with the API?

try {
    URL url = new URL("https://southeastasia.api.cognitive.microsoft.com/face/v1.0/verify?faceId1=" + face1 + "&faceId2=" + face2);
    HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
    urlConnection.setRequestProperty("Ocp-Apim-Subscription-Key", "----");
    try {
        BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
        StringBuilder stringBuilder = new StringBuilder();
        String line;
        while ((line = bufferedReader.readLine()) != null) {
            stringBuilder.append(line).append("\n");
        }
        bufferedReader.close();
        getting = stringBuilder.toString();
        Log.v("Gettinganswer", getting);
        return  getting;
    } finally {
        urlConnection.disconnect();
    }
}
catch(Exception e) {
    Log.e("ERROR", e.getMessage(), e);
    return null;
}

2018-10-24 23:45:44.200 25987-26725/com.example.android.staysaferesq E/ERROR: https://southeastasia.api.cognitive.microsoft.com/face/v1.0/verify?faceId1=03ab5c58-86bb-4eef-a89e-56d4ce76e95b&faceId2=03ab5c58-86bb-4eef-a89e-56d4ce76e95b
    java.io.FileNotFoundException: https://southeastasia.api.cognitive.microsoft.com/face/v1.0/verify?faceId1=03ab5c58-86bb-4eef-a89e-56d4ce76e95b&faceId2=03ab5c58-86bb-4eef-a89e-56d4ce76e95b
        at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:251)
        at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210)
        at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(Unknown Source:0)
        at com.example.android.staysaferesq.FaceActivity$RetrieveFeedTask.doInBackground(FaceActivity.java:301)
        at com.example.android.staysaferesq.FaceActivity$RetrieveFeedTask.doInBackground(FaceActivity.java:275)
        at android.os.AsyncTask$2.call(AsyncTask.java:333)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:245)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
        at java.lang.Thread.run(Thread.java:764)

Answer:

Does this have something to do with the API?

According to the document we could know the face verify API endpoint should be

post https://[location].api.cognitive.microsoft.com/face/v1.0/verify

The body format is

{
 "faceId1": "xxxxxxxx",
 "faceId2": "xxxxx",
}

Question:

I have a Data in Azure Storage (Table), I take that Data and serialize it to Json format throw API, how can I parse it in Java (Android Studio) in Collection of my class?

public class Book {
String name;
String author;
String description;

public Book(){}

public Book(String name, String author, String description) {
    this.name = name;
    this.author = author;
    this.description = description;
   }
}

JSON which API will give:

"{\"operations\":[{\"title\":\"1984\",\"author\":\"Oruel\",\"description\":\"Antiutopy\",\"PartitionKey\":\"Books\",\"RowKey\":\"\",\"Timestamp\":\"2017-03-10T19:23:49.6668457+00:00\",\"ETag\":\"W/\\\"datetime'2017-03-10T19%3A23%3A49.6668457Z'\\\"\"},{\"title\":\"hello\",\"author\":\"world\",\"description\":\"test\",\"PartitionKey\":\"Books\",\"RowKey\":\"32de9806-617a-4b66-9a87-bd3488e9080e\",\"Timestamp\":\"2017-03-08T12:32:45.2881957+00:00\",\"ETag\":\"W/\\\"datetime'2017-03-08T12%3A32%3A45.2881957Z'\\\"\"}]}"

Answer:

It seems that you want to retrieve data from Azure Table Storage and using Java on Android.

The simple way is that using Azure Storage SDK for Android and refer to the offical tutorial to do these operations via rewrite your Book class which inherit TableServiceEntity as below. For more details, please see the javadocs of Azure Storage Android SDK.

public class Book extends TableServiceEntity {
    String name;
    String author;
    String description;

    public Book(){}

    public Book(String sn, String name, String author, String description) {
        this.partitionKey = "Books";
        this.rowKey = sn;
        this.name = name;
        this.author = author;
        this.description = description;
    }

    // Setter & Getter methods for properties
    ..............
}

If you just need to serialize & deserialize the data between JSON and POJO Entity, please use Gson library to do it.

Question:

I'm trying to create a schema for hive to parse json, however, I am having trouble creating the schema when the json doc is in the following structure:

{
    "context": {        
        "custom": {
            "dimensions": [{
                "action": "GetFilters"
            },
            {
                "userId": "12345678"
            }]
        }
    }
}

I am using the Hadoop emulator for Azure's HDInsights on windows (8.1) and am using java (1.8.0_73). I compiled the SerDe successfully with Maven. I would think that the following would work:

add jar ../lib/json-serde-1.1.9.9-Hive1.2-jar-with-dependencies.jar;

DROP TABLE events;
CREATE EXTERNAL TABLE events (
  context STRUCT<custom:STRUCT<dimensions:array<STRUCT<action:string>,STRUCT<userId:string>>>>
) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
LOCATION '/json/event';

When I take out the nested ARRAY>, then the schema parses ok, but with it in, I get the following exception:

MismatchedTokenException(282!=9) at org.antlr.runtime.BaseRecognizer.recoverFromMismatchedToken(BaseRecog nizer.java:617) at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115) at org.apache.hadoop.hive.ql.parse.HiveParser.columnNameColonType(HivePa rser.java:34909) at org.apache.hadoop.hive.ql.parse.HiveParser.columnNameColonTypeList(Hi veParser.java:33113) at org.apache.hadoop.hive.ql.parse.HiveParser.structType(HiveParser.java :36331) at org.apache.hadoop.hive.ql.parse.HiveParser.type(HiveParser.java:35334 ) at org.apache.hadoop.hive.ql.parse.HiveParser.colType(HiveParser.java:35 054) at org.apache.hadoop.hive.ql.parse.HiveParser.columnNameColonType(HivePa rser.java:34914) at org.apache.hadoop.hive.ql.parse.HiveParser.columnNameColonTypeList(Hi veParser.java:33085) at org.apache.hadoop.hive.ql.parse.HiveParser.structType(HiveParser.java :36331) at org.apache.hadoop.hive.ql.parse.HiveParser.type(HiveParser.java:35334 ) at org.apache.hadoop.hive.ql.parse.HiveParser.colType(HiveParser.java:35 054) at org.apache.hadoop.hive.ql.parse.HiveParser.columnNameType(HiveParser. java:34754) at org.apache.hadoop.hive.ql.parse.HiveParser.columnNameTypeList(HivePar ser.java:32951) at org.apache.hadoop.hive.ql.parse.HiveParser.createTableStatement(HiveP arser.java:4544) at org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.ja va:2144) at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.j ava:1398) at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java: 1036) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:19 9) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:16 6) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:409) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:323) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:980) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1045) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:2 68) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793 ) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) FAILED: ParseException line 2:69 missing > at ',' near 'STRUCT' in column specif ication line 2:76 mismatched input '<' expecting : near 'STRUCT' in column specification

hive>

Answer:

That external table looked good to me. Maybe try downloading another distribution of that JSON serde. I have had success with: http://www.congiu.net/hive-json-serde/

I have had success in HDInsight 3.2 with http://www.congiu.net/hive-json-serde/1.3/cdh5/ but you might try a newer build for HDP.

Documentation here: https://github.com/rcongiu/Hive-JSON-Serde

Question:

I'm using the Azure SDK which includes GSON to store query results from Microsoft Azure into a class. I've been following the instructions from this example: http://blogs.msdn.com/b/carlosfigueira/archive/2013/06/19/custom-api-in-azure-mobile-services-client-sdks.aspx

So I was able to save a simple example into a class of all Strings. However I hit a roadblock when trying to store data into a class with mixed types including an ArrayList. Here's my class:

class clsUser {
    public String UserID;
    public String UserName;
    public String UserStatus;
    public ArrayList<String> UserEmails;
    //Constructors, methods, etc
}

An instance of clsUser can have 0 or many UserEmails, so I use an ArrayList. I should say that I've been using this class in my app - pulling JSON data is new, but I'd like to go this route.

The JsonObject looks like this:

{"UserID":85,"UserName":null,"UserEmails":"Call.Me@Maybe.org","UserStatus":"T"}

I've also tried adjusting the API so it returns each of these:

{"UserID":85,"UserName":null,"UserEmails":"[Call.Me@Maybe.org]","UserStatus":"T"}

{"UserID":85,"UserName":null,"UserEmails":"[{Call.Me@Maybe.org}]","UserStatus":"T"}

And I'm getting an error "Expected BEGIN_ARRAY but was STRING" which I know means it sees "Call.Me@Maybe.org" as a String, but how can I make it go into the ArrayList field?:

07-31 22:59:18.776  27842-27842/com.myapp.app.debug E/AndroidRuntime﹕ FATAL EXCEPTION: main
    Process: com.myapp.app.debug, PID: 27842
    com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_ARRAY but was STRING
            at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:176)
            at com.google.gson.Gson.fromJson(Gson.java:795)
            at com.google.gson.Gson.fromJson(Gson.java:859)
            at com.google.gson.Gson.fromJson(Gson.java:832)
            at com.microsoft.windowsazure.mobileservices.JsonEntityParser.parseResults(JsonEntityParser.java:62)
            at com.microsoft.windowsazure.mobileservices.MobileServiceClient$4.onCompleted(MobileServiceClient.java:615)
            at com.microsoft.windowsazure.mobileservices.MobileServiceClient$5.onResponse(MobileServiceClient.java:710)
            at com.microsoft.windowsazure.mobileservices.MobileServiceClient$6.onPostExecute(MobileServiceClient.java:825)
            at com.microsoft.windowsazure.mobileservices.MobileServiceClient$6.onPostExecute(MobileServiceClient.java:1)
            at android.os.AsyncTask.finish(AsyncTask.java:632)
            at android.os.AsyncTask.access$600(AsyncTask.java:177)
            at android.os.AsyncTask$InternalHandler.handleMessage(AsyncTask.java:645)
            at android.os.Handler.dispatchMessage(Handler.java:102)
            at android.os.Looper.loop(Looper.java:146)
            at android.app.ActivityThread.main(ActivityThread.java:5487)
            at java.lang.reflect.Method.invokeNative(Native Method)
            at java.lang.reflect.Method.invoke(Method.java:515)
            at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1283)
            at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1099)
            at dalvik.system.NativeStart.main(Native Method)
     Caused by: java.lang.IllegalStateException: Expected BEGIN_ARRAY but was STRING
            at com.google.gson.internal.bind.JsonTreeReader.expect(JsonTreeReader.java:139)
            at com.google.gson.internal.bind.JsonTreeReader.beginArray(JsonTreeReader.java:58)
            at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:79)
            at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:60)
            at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
            at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
            at com.google.gson.Gson.fromJson(Gson.java:795)
            at com.google.gson.Gson.fromJson(Gson.java:859)
            at com.google.gson.Gson.fromJson(Gson.java:832)
            at com.microsoft.windowsazure.mobileservices.JsonEntityParser.parseResults(JsonEntityParser.java:62)
            at com.microsoft.windowsazure.mobileservices.MobileServiceClient$4.onCompleted(MobileServiceClient.java:615)
            at com.microsoft.windowsazure.mobileservices.MobileServiceClient$5.onResponse(MobileServiceClient.java:710)
            at com.microsoft.windowsazure.mobileservices.MobileServiceClient$6.onPostExecute(MobileServiceClient.java:825)
            at com.microsoft.windowsazure.mobileservices.MobileServiceClient$6.onPostExecute(MobileServiceClient.java:1)
            at android.os.AsyncTask.finish(AsyncTask.java:632)
            at android.os.AsyncTask.access$600(AsyncTask.java:177)
            at android.os.AsyncTask$InternalHandler.handleMessage(AsyncTask.java:645)
            at android.os.Handler.dispatchMessage(Handler.java:102)
            at android.os.Looper.loop(Looper.java:146)
            at android.app.ActivityThread.main(ActivityThread.java:5487)
            at java.lang.reflect.Method.invokeNative(Native Method)
            at java.lang.reflect.Method.invoke(Method.java:515)
            at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1283)
            at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1099)
            at dalvik.system.NativeStart.main(Native Method)

I think the error is either in the data or the class, but here's the code I'm using for the sake of completeness. None of the logs get a chance to run before the error comes up:

List<Pair<String, String>> params = new ArrayList<Pair<String, String>>();
params.add(new Pair<String, String>("email", emailtext));
params.add(new Pair<String, String>("token", Token));
Azure.invokeApi("user/login", "GET", params, clsUser.class, new ApiOperationCallback<clsUser>() {
    @Override
    public void onCompleted(clsUser result, Exception e, ServiceFilterResponse response) {
        if (e == null) {
            Log.d("Login GetToken","Azure success: ID=" + result.getUserID());
            Log.d("Login GetToken","Azure success: " + result.getAll().toString());
            currentUser = result;
        } else {
            Log.d("Login GetToken","Azure Error: " + e.toString());
        }
    }
});

Anyone have experience with this kind of mixed class?


Answer:

I haven't tested this for this situation (since August I've come up with a workaround that I don't feel like revisiting).

However I think the key here is to create a custom deserializer to tell Gson how to interpret the data.

See: Custom JSON deserializer using Gson

Then use mClient.registerSerializer method

Question:

I'm getting the following error on my android logcat when I try to login.

1492-1639/net.azurewebsites.cosy W/System.err﹕ org.json.JSONException: Value The of type java.lang.String cannot be converted to JSONObject

I'm definitely getting requests to my server so I don't know if it's the value it's returning or is it a problem in the app and it can't convert it.

my php is the following:

if( $conn ) {
    $username = $_POST["username"];
    $password = $_POST["password"];
    //echo($username);
    // echo "Connection established.<br />";
    $query = sprintf("SELECT * from Users where username = ? and password = ?");
    $params1 = array( $username, $password);
    $stmt = sqlsrv_query($conn, $query, $params1);
    if ($stmt === false) {
        die(print_r(sqlsrv_errors(), true));
    }
    $Users = array();
    while ($row = sqlsrv_fetch_array($stmt, SQLSRV_FETCH_ASSOC)) {
        $username1 = $row['username'];
        $password1= $row['password'];
    }
    $Users["username"] = $username1;
    $Users["password"] = $password1;
    echo json_encode($Users);
    sqlsrv_free_stmt($stmt);
    sqlsrv_close( $conn );
} else {
    echo "Connection could not be established.<br />";
    die( print_r( sqlsrv_errors(), true));
}

And my request consists of the following:

public class ServerRequests {
    ProgressDialog progressDialog;
    public static final int CONNECTION_TIMEOUT = 1000 * 15;
    public static final String SERVER_ADDRESS = "http://cosy.azurewebsites.net/";

    public ServerRequests(Context context) {
        progressDialog = new ProgressDialog(context);
        progressDialog.setCancelable(false);
        progressDialog.setTitle("Processing...");
        progressDialog.setMessage("Please wait...");
    }

    public void fetchUserDataAsyncTask(User user, GetUserCallback userCallBack) {
        progressDialog.show();
        new fetchUserDataAsyncTask(user, userCallBack).execute();
    }

    public class fetchUserDataAsyncTask extends AsyncTask<Void, Void, User> {
        User user;
        GetUserCallback userCallBack;

        public fetchUserDataAsyncTask(User user, GetUserCallback userCallBack) {
            this.user = user;
            this.userCallBack = userCallBack;
        }

        @Override
        protected User doInBackground(Void... params) {
            ArrayList<NameValuePair> dataToSend = new ArrayList<>();
            dataToSend.add(new BasicNameValuePair("username", user.username));
            dataToSend.add(new BasicNameValuePair("password", user.password));

            HttpParams httpRequestParams = new BasicHttpParams();
            HttpConnectionParams.setConnectionTimeout(httpRequestParams, CONNECTION_TIMEOUT);
            HttpConnectionParams.setSoTimeout(httpRequestParams, CONNECTION_TIMEOUT);

            HttpClient client = new DefaultHttpClient(httpRequestParams);
            HttpPost post = new HttpPost(SERVER_ADDRESS + "FetchUserData.php");

            User returnedUser = null;

            try {
                post.setEntity(new UrlEncodedFormEntity(dataToSend));
                HttpResponse httpResponse = client.execute(post);

                HttpEntity entity = httpResponse.getEntity();
                String result = EntityUtils.toString(entity);
                JSONObject jObject = new JSONObject(result);

                if (jObject.length() != 0) {
                    Log.v("happened", "2");
                    returnedUser = new User(user.username, user.password);
                }

            } catch (Exception e) {
                e.printStackTrace();
            }

            return returnedUser;
        }

        @Override
        protected void onPostExecute(User returnedUser) {
            super.onPostExecute(returnedUser);
            progressDialog.dismiss();
            userCallBack.done(returnedUser);
        }
    }
}

I have spent forever on this and I just can't find out where I am going wrong any help at all would be appreciated!!


Answer:

It seems to be that the PHP response string contains some special characters or messy codes. So the org.json library in Java can not convert the response string to a JSONObject.

You can try to use the functions urlencode & urldecode in PHP to deal with the string. Please see the codes below.

Modify the codes below.

while ($row = sqlsrv_fetch_array($stmt, SQLSRV_FETCH_ASSOC)) {
    $username1 = urlencode($row['username']);
    $password1= urlencode($row['password']);
}

And

echo urldecode(json_encode($Users));

Hope it helps. Any concern, please feel free to let me know.