Languages

Menu
Sites
Language
JSON response truncated when request a huge data using CURL

 

I'm making a simple http request using CURL using getinmemory.c from CURL example page. There is no issue happen when requesting a single post json data. But, when requesting a large json data, the JSON result has truncated or receviced partially. Here is my code :

struct MemoryStruct {
        char *memory;
	size_t size;
};

static size_t WriteMemoryCallback(void *contents, size_t size, size_t nmemb, void *userp) {
	size_t realsize = size * nmemb;
	struct MemoryStruct *mem = (struct MemoryStruct *) userp;

	mem->memory = realloc(mem->memory, mem->size + realsize + 1);
	if (mem->memory == NULL) {
		/* out of memory! */
		printf("not enough memory (realloc returned NULL)\n");
		return 0;
	}

	memcpy(&(mem->memory[mem->size]), contents, realsize);
	mem->size += realsize;
	mem->memory[mem->size] = 0;

	return realsize;
}

static void ExecuteCurl() {
	CURL *curl_handle;
	CURLcode res;

	struct MemoryStruct chunk;

	chunk.memory = malloc(sizeof(char*)); /* will be grown as needed by the realloc above */
	chunk.size = 0; /* no data at this point */

	curl_global_init(CURL_GLOBAL_ALL);

	/* init the curl session */
	curl_handle = curl_easy_init();

	if (curl_handle) {
		/* specify URL to get */
		curl_easy_setopt(curl_handle, CURLOPT_URL, "https://jsonplaceholder.typicode.com/posts");
                //curl_easy_setopt(curl_handle, CURLOPT_URL, "https://jsonplaceholder.typicode.com/posts/1");

		/* send all data to this function  */
		curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);

		/* we pass our 'chunk' struct to the callback function */
		curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, (void * )&chunk);

		/* some servers don't like requests that are made without a user-agent
		 field, so we provide one */
		curl_easy_setopt(curl_handle, CURLOPT_USERAGENT, "libcurl-agent/1.0");

		/* get it! */
		res = curl_easy_perform(curl_handle);

		/* check for errors */
		if (res != CURLE_OK) {
			dlog_print(DLOG_ERROR, "CURL", "curl_easy_perform() failed: %s\n", curl_easy_strerror(res));

		} else {
			/*
			 * Now, our chunk.memory points to a memory block that is chunk.size
			 * bytes big and contains the remote file.
			 *
			 * Do something nice with it!
			 */

			dlog_print(DLOG_DEBUG, "CURL", "Bytes received: %d\n", strlen(chunk.memory));
			dlog_print(DLOG_DEBUG, "CURL", "JSON: %s\n", chunk.memory);
		}

		/* cleanup curl stuff */
		curl_easy_cleanup(curl_handle);

		free(chunk.memory);
	}

	/* we're done with libcurl, so clean it up */
	curl_global_cleanup();
}

FYI: I'm creating this project using Online Sample Project - Basic UI with EDC, then by adding code above I'm executing ExecuteUrl() method within app_create(void *data).

Is there any other way to handle large json response data? Thanks.

Regards.

Responses

1 Replies
Shaswati Saha

Hello,

Please try to implement this by following the procedure in the below link:

https://gist.github.com/leprechau/e6b8fef41a153218e1f4

Here, the same thing is done as yours. Hope, it may help!

Good Luck!