IG auto logged me out #45

fifson1 · 2022-07-27T23:12:01Z

I have two errors with autofetcher.
First when I want run script I get this error:
SyntaxError: Unexpected token < in JSON at position 0

I changed data in Logs section but after this I get another error
Exception: Attribute provided with invalid value: Header:x-ig-www-claim

How I can fix this?

chriskyfung · 2022-07-28T04:11:23Z

The error SyntaxError: Unexpected token < in JSON at position 0 raised because it failed to parse response from Instagram API. It may due to pass a wrong Instagram user ID.

chriskyfung · 2022-07-28T04:12:55Z

Regarding to Exception: Attribute provided with invalid value: Header:x-ig-www-claim, please go to Issue #41

fifson1 · 2022-07-28T10:26:33Z

The error SyntaxError: Unexpected token < in JSON at position 0 raised because it failed to parse response from Instagram API. It may due to pass a wrong Instagram user ID.

No, id is good. I started use your project from april and since 26 July starts show this error. Before everything was good.

Regarding to Exception: Attribute provided with invalid value: Header:x-ig-www-claim, please go to Issue #41

Yes, I read this. But I don't know how and where change this settings in your code. Can you write step by step how fix this error?

fifson1 · 2022-07-28T11:36:21Z

The error SyntaxError: Unexpected token < in JSON at position 0 raised because it failed to parse response from Instagram API. It may due to pass a wrong Instagram user ID.

I copy new version google sheet and copy all data in setting. Run works but when I log out from instagram this error show again and not work. After log in again in instagram x-ig-www-claim and cookie changed.

chriskyfung · 2022-07-29T06:28:01Z

The error SyntaxError: Unexpected token < in JSON at position 0 raised because it failed to parse response from Instagram API. It may due to pass a wrong Instagram user ID.

I copy new version google sheet and copy all data in setting. Run works but when I log out from instagram this error show again and not work. After log in again in instagram x-ig-www-claim and cookie changed.

You should never log out from Instagram because the script use the cookie of that log-in session.

fifson1 · 2022-07-29T10:25:38Z

Another error

Exception: Request failed for https://i.instagram.com returned code 400. Truncated server response: {"message":"checkpoint_required","checkpoint_url":"https://i.instagram.com/challenge/?next=/api/v1/feed/reels_media/...",... (use muteHttpExceptions option to examine full response)

I check and Instagram block my account. I try with another one, I don't log out. I hope this time it works.

fifson1 · 2022-08-07T11:59:07Z

Ok, I tested script and it works but after 2 days instagram block my accout, I must authorized. After authorized is error "SyntaxError: Unexpected token < in JSON at position 0" and I must copy new data to settings. Any suggestion how fix it? I want copy data once not new after 2 days.

Maybe I must change trigger? Now I have set run after every 1 hour. Maybe try set after longer interval?

fifson1 · 2022-08-10T02:52:47Z

Even when autorize error don't appear on Instagram I must copy new cookies date afer 2-3 days, because "SyntaxError: Unexpected token < in JSON at position 0" appear when I run fetcher. Any suggestion how fix this? I dont log out from IG account. I open Firefox - log in to IG - and close Firefox. I dont use Firefox to check sites, I using Chrome.

chriskyfung · 2022-08-10T15:12:55Z

@fifson1 Why don't you copy your cookies from Chrome 😅

fifson1 · 2022-08-10T18:08:03Z

@fifson1 Why don't you copy your cookies from Chrome 😅

I don't want be log in on Chrome, but if this resolve error I could try
And how should work testPipeline? When I run testPipeline then give me info: "Number of downloadable media files from @bbcnews: 16" but problem is I don't have @bbcnews in my sheet.

fifson1 · 2022-08-13T12:20:06Z

I think IG auto log out after 3 days and when I must log in again cookies is changed and new cookies must be copy to sheet because show error "SyntaxError: Unexpected token < in JSON at position 0", now I try with Chrome, maybe will be better effect

fifson1 · 2022-08-26T11:05:29Z

Still same problems with this ;/ IG logged me out after 3-4 days and I must log in and copy new cookies again and again. Maybe is something bad with my chrome settings? I have no idea what fix this...

chriskyfung · 2022-08-26T14:05:26Z

Instagram enhanced their endpoint security recently. It seems the Instagram team is going to prevent robots to access their API just using cookies. They become verify user sessions and credentials more frequently. Thus, the problem won't fix very soon until rewriting entire piece of code to use username and password for authentication.

fifson1 · 2022-08-26T17:19:47Z

Instagram enhanced their endpoint security recently. It seems the Instagram team is going to prevent robots to access their API just using cookies. They become verify user sessions and credentials more frequently. Thus, the problem won't fix very soon until rewriting entire piece of code to use username and password for authentication.

So only way for now is wait for re-build code, because others attempts to solve problem don't work? I set trigger to get email notifs when error apear so I know when IG log me out and sheet not work and I can fast fix this, but would be better copy cookies once on month and don't fix error after few days. But most important is fact that main task in project works correct and files is saved in GDrive

fifson1 · 2022-09-17T23:11:00Z

New relese from 2022-09-04 can help with problems with log out? I don't know make change or not...

chriskyfung · 2022-09-18T09:04:37Z

New relese from 2022-09-04 can help with problems with log out? I don't know make change or not...

No, it doesn't fix that issue.

chriskyfung · 2022-10-11T03:48:23Z

I found Instagram won't require re-login and change the cookie every day if I remove the time-based trigger from the Apps Script project and only run the script manually once per day.

fifson1 · 2022-10-12T10:18:53Z

I found Instagram won't require re-login and change the cookie every day if I remove the time-based trigger from the Apps Script project and only run the script manually once per day.

I will test how many max. run can be before log out. Now I have 2 sheets, in both I set trigger run after 2 hours.

Seeker220 · 2022-10-12T14:27:14Z

I found Instagram won't require re-login and change the cookie every day if I remove the time-based trigger from the Apps Script project and only run the script manually once per day.

I faced the same logout problem more than one year back, when i scheduled the script to run every 8 hours. After being banned by insta twice, and thinking that only a robot could request exactly every 8 hours, I changed the coded to run the trigger in a random time between every 8th and 9th hour:

function getRandomIntInclusive(min, max) {
  min = Math.ceil(min);
  max = Math.floor(max);
  return Math.floor(Math.random() * (max - min + 1) + min); //The maximum is inclusive and the minimum is inclusive
}


function Triggerthis() {
  const functionName = 'run';
  const miliseconds = getRandomIntInclusive(60000, 360000)

  ScriptApp.newTrigger(functionName)
  .timeBased()
  .after(miliseconds)
  .create();
}

This worked nicely until recently. I think Instagram now has deployed other security modifications :(

P.S: @chriskyfung plz fix this :) This is the only auto instagram story downloader on the internet...

chriskyfung · 2022-10-12T16:01:01Z

@Seeker220 Did you try a larger interval, such as 12-24 hours?

Seeker220 · 2022-10-12T16:43:07Z

OK done.. I will observe and reply tomorrow :)

chriskyfung · 2022-10-12T16:56:11Z

@Seeker220 Please update and test with the new v6 template, and let me know your result.

I added the param x-instagram-ajax to the header as in a browser, but it still have the logout problem when I used a time-based trigger a couple weeks ago. As it doesn't work as expected, so far I didn't merged the changes to the master branch.

I think there is no solution by using just the cookies in the future. It will need a headless browser, such as Selenium and Puppeteer to mimic the login process to work around. However, it will unable to build with just Apps Script. Each user may need to his/her own server or cloud function to run those third-party programs securely.

Seeker220 · 2022-10-12T22:01:38Z

@chriskyfung
v6 gives the following error:

Seeker220 · 2022-10-12T22:10:13Z

I made a single all-in-one script today... giving it:

function getRandomIntInclusive(min, max) {
  min = Math.ceil(min);
  max = Math.floor(max);
  return Math.floor(Math.random() * (max - min + 1) + min); //The maximum is inclusive and the minimum is inclusive
}


function Trigger() {
  const functionName = 'run';
  const miliseconds = getRandomIntInclusive(60000, 3600000)

  ScriptApp.newTrigger(functionName)
  .timeBased()
  .after(miliseconds)
  .create();
}






// User-defined Variables
var g_username = 'username';
var g_password = 'password';
var gdrive_id = 'GOOGLE_DRIVE_FOLDER_ID';
var lastlog_id = '1XXXXXXXXXX_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX';
var historylog_id = '1XXXXXXXXXX_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX';
var fetchContentLog_id = '1XXXXXXXXXX_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX';
var statusBadge_id = '1XXXXXXXXXX_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX';
var lastTestedBadge_id = '1XXXXXXXXXX_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX';
var crashReportEmail = '[email protected]';

var COOKIE = 'cookie';
var X_ASBD_ID = '123456';
var X_CSRFTOKEN = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX';
var X_IG_APP_ID = '123456780000000';
var X_IG_WWW_CLAIM = 'hmac.XXXXXXXXXXXXXX-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX';
var X_INSTAGRAM_AJAX = '1000000000';

var isDebug = false;

function getQuery(ig_user_id){  
  return "https://i.instagram.com/api/v1/feed/reels_media/?reel_ids=" + ig_user_id;
}

function fetch_ig_stories(query){
  var opt3 = {
    "headers": {
      "accept": "*/*",
      "cache-control": "no-cache",
      "pragma": "no-cache",
      "sec-fetch-dest": "empty",
      "sec-fetch-mode": "cors",
      "sec-fetch-site": "same-site",
      "x-ig-app-id": X_IG_APP_ID,
      "x-ig-www-claim": X_IG_WWW_CLAIM,
      "x-asbd-id": X_ASBD_ID,
      "x-csrftoken": X_CSRFTOKEN,
      "x-instagram-ajax": X_INSTAGRAM_AJAX,
      "cookie": COOKIE
    },
    "referrer": "https://www.instagram.com/",
    "referrerPolicy": "strict-origin-when-cross-origin",
    "body": null,
    "method": "GET",
    "mode": "cors"
  };
  var data = UrlFetchApp.fetch(query, opt3).getContentText();
  if (isDebug) { Logger.log(data); }
  return data;
}

function parseDownloadUrl(html,isTest){
  if (html == '{"reels": {}, "reels_media": [], "status": "ok"}') {
    return [];
  };
  if (isTest) {
    var data = JSON.parse(html);
  } else {
    var fetchContentlog = DocumentApp.openById(fetchContentLog_id);
    fetchContentlog.clear();
    fetchContentlog.appendParagraph(html);
    var body = fetchContentlog.getBody().getText();
    var data = JSON.parse(body);
  }
  var items = data.reels_media[0].items;

  var urls = [];
  for (i in items) {
    var item = items[i];
    if (item.video_versions) {
      var videos = item.video_versions;
      urls[i] = videos[0].url;
    } else {
      urls[i] = item.image_versions2.candidates[0].url;
    }
  }
  if (isDebug) { Logger.log(urls); }
  return urls;
}

function __main__(targetIgUser,gdrive_id) {
  
  // Access the body of the log google doc file
  var d = new Date();
  var lastlogfile = DocumentApp.openById(lastlog_id);
  var oldfile = lastlogfile.getBody().copy(); //backup the previous log 
  lastlogfile.clear(); // clear the log file for logging current process
  lastlogfile.getBody().appendParagraph(targetIgUser.name + "\t" + d); // append date and time to the log file
  
  // Fetch URL
  var query_url = getQuery(targetIgUser.id);
  var html = fetch_ig_stories(query_url);
  var urls = parseDownloadUrl(html,false);
  
  var msg = '';
  
  // For each article element found in the HTML
  if (urls != null && urls.length > 0) {
    for (i in urls) {
      var url = urls[i];
      
      // log the url found to the Doc file
      lastlogfile.getBody().appendParagraph('');
      lastlogfile.getBody().appendParagraph(url);
      
      // check DL history
      var u0 = url.split("?")[0];
      var hasDL = oldfile.findText(u0);      
      if (!hasDL) { // download the file to Google Drive
        msg = uploadToDrive(url, gdrive_id , '');      
      } 
      else {
        msg = 'File has already been fetched once.'
      }
      lastlogfile.getBody().appendParagraph('\n\t' + msg); // log the upload msg to the Doc file
      Logger.log(msg);            
    }    
  }  
  var logtxt = lastlogfile.getBody().getText();
  var historyfile = DocumentApp.openById(historylog_id);
  historyfile.getBody().appendParagraph(logtxt);
  
  return logtxt;  
}

function doGet(e,gfolderid) {
  
  var usr = '';
  var pwd = '';
  var target = '';
  
  // parse query
  try {
    usr = e.parameter.usr.trim();
    pwd = e.parameter.pwd.trim();
    target = (typeof(e.parameter.target) === 'string') ? JSON.parse(e.parameter.target) : e.parameter.target;
  }
  catch(err) {
    Logger.log(err);
  }
  
  var msg = '';
  
  // execute if correct username and password in the query
  if (usr == g_username && pwd == g_password && target != '') {
    msg = __main__(target,gfolderid);
    msg = __main__(target,gfolderid);
    msg = ContentService.createTextOutput(msg);
  }
  else { msg = ContentService.createTextOutput('Invalid username and password or targetID!');
        Logger.log(msg);
  };  
  return msg;
}

function story_get(targetusername,userid,savelocation) {
    var e = {
      "parameter": {
        "usr": g_username,
        "pwd": g_password,
        "target" : { "name": targetusername, "id": userid}
        //"target" : { "name": "medium", "id":"1112881921"}
      }
    };
  doGet(e,savelocation);
}

function u1() {
  try{
    story_get("username1","id1","folder1");
  }
  catch(err) {
    Logger.log(err);
  }
}
function u2() {
  try{
    story_get("username2","id2","folder2");
  }
  catch(err) {
    Logger.log(err);
  }
}
function u3() {
  try{
    story_get("username3","id3","folder3");
  }
  catch(err) {
    Logger.log(err);
  }
}

function run() {
  u1();
  u2();
  u3();
}

and set trigger each 12 hours to the function Trigger()

chriskyfung · 2022-10-13T06:07:20Z

@chriskyfung v6 gives the following error:

Please name cell B8 to X_INSTAGRAM_AJAX on the Settings page as below.

chriskyfung · 2022-10-13T06:17:03Z

I made a single all-in-one script today... giving it:
function getRandomIntInclusive(min, max) {
              ⋮
              ⋮
and set trigger each 12 hours to the function Trigger()

@Seeker220 DO NOT post any sensitive personal data, please remember to remove them from the code next time.

Seeker220 · 2022-10-13T06:23:58Z

I made a single all-in-one script today... giving it:
function getRandomIntInclusive(min, max) {
              ⋮
              ⋮
and set trigger each 12 hours to the function Trigger()
@Seeker220 DO NOT post any sensitive personal data, please remember to remove them from the code next time.

Actually I changed the values a little bit :) just gave them for format...

Seeker220 · 2022-10-13T06:55:03Z

@chriskyfung I opened a new insta account for testing, and set v6 trigger 12 hours. I noticed that in both my accounts, "X_ASBD_ID" and "X_IG_APP_ID" has the same value, although opened one in Chrome and one in Firefox...

fifson1 · 2022-10-17T11:13:21Z

@chriskyfung I opened a new insta account for testing, and set v6 trigger 12 hours. I noticed that in both my accounts, "X_ASBD_ID" and "X_IG_APP_ID" has the same value, although opened one in Chrome and one in Firefox...

@Seeker220 any details after testing? IG still log out?

Seeker220 · 2022-10-17T11:28:11Z

@chriskyfung I opened a new insta account for testing, and set v6 trigger 12 hours. I noticed that in both my accounts, "X_ASBD_ID" and "X_IG_APP_ID" has the same value, although opened one in Chrome and one in Firefox...

@Seeker220 any details after testing? IG still log out?

@fifson1 no, IG did not log out as far. :)

fifson1 · 2022-10-20T17:52:15Z

New problem today. IG don't log me out from yesterday but when trigger or I run script I have error:
"Error: Request failed for https://i.instagram.com/ returned code 429. Truncated server response: {"message":"","spam":true,"status":"fail"} (use muteHttpExceptions option to examine full response) (error code: 0xf1)"
After 5-6 times press run finally script work and saves files.
Any idea how fix this? I use ver.5 not with library v6, you think better change this and use newest version?
edit. I try with new version and still same error. Maybe too many instagram acc?

chriskyfung · 2022-10-22T07:52:27Z

New problem today. IG don't log me out from yesterday but when trigger or I run script I have error: "Error: Request failed for https://i.instagram.com/ returned code 429. Truncated server response: {"message":"","spam":true,"status":"fail"} (use muteHttpExceptions option to examine full response) (error code: 0xf1)" ...

I have no response to fix this issue if you overuse the script and get blocked by Instagram. Please investigate the problem yourself.

fifson1 · 2022-10-22T13:57:33Z

New problem today. IG don't log me out from yesterday but when trigger or I run script I have error: "Error: Request failed for https://i.instagram.com/ returned code 429. Truncated server response: {"message":"","spam":true,"status":"fail"} (use muteHttpExceptions option to examine full response) (error code: 0xf1)" ...

I have no response to fix this issue if you overuse the script and get blocked by Instagram. Please investigate the problem yourself.

I made new IG account and copy cookies from this, for now script work.

gatoradez · 2023-03-21T18:16:57Z

Hi, I tried the hourly (12h), daily and weekly trigger but Instagram logged out me every time.

Error: Request failed for https://i.instagram.com returned code 401. Truncated server response: {"message":"請幾分鐘後再試一次。","require_login":true,"status":"fail"} (use muteHttpExceptions option to examine full response) (error code: 0xf1)

Can you please confirm that currently - March 2023 - the only solution provided to make the tool work properly is TO DO NOT set the trigger? So the only solution is to run manually the task (button)? If this is correct, it should be removed the trigger from the script because it is misleading, isn't?
Or are there some workarounds?

chriskyfung · 2023-03-24T15:43:14Z

Instagram forces log out even running the script manually.

You need to open your Instagram account using the Chrome browser once a day!

Sylphystia · 2023-08-24T09:04:31Z

Instagram seems to be logging me out every time I run the script before it can actually save anything, rendering the script useless. Is there a workaround for this at all? Am I just unlucky or did IG finetune their detection of using cookies to access their API?

chriskyfung · 2024-02-04T01:39:37Z

Updates

In Library v9, error logging will be improved to include HTTP headers.

Feb 4, 2024, 9:30:04 AM	Warning	HTTP headers: undefined
Feb 4, 2024, 9:30:04 AM	Error	Error: Request failed for https://i.instagram.com returned code 400. Truncated server response: {"message":"checkpoint_required","checkpoint_url":"https://i.instagram.com/challenge/?next=/api/v1/feed/reels_media/%253Freel_ids%253D16278726","lo... (use muteHttpExceptions option to examine full response) (code: 0xf1)
    at getInstagramData(bundle:608:11)
    at fetch(bundle:687:16)
    at [unknown function](bundle:877:17)
    at batchFetch(bundle:875:8)
    at run(code:3:8)

chriskyfung added the question Further information is requested label Jul 28, 2022

fifson1 changed the title ~~Two errors~~ IG auto logged me out Aug 26, 2022

chriskyfung added the help wanted Extra attention is needed label Aug 26, 2022

chriskyfung added a commit that referenced this issue Sep 8, 2022

Add response type error check #45

85dfa99

chriskyfung closed this as completed Sep 18, 2022

chriskyfung reopened this Sep 19, 2022

chriskyfung closed this as completed in 1ccd681 Oct 12, 2022

chriskyfung reopened this Oct 12, 2022

chriskyfung mentioned this issue Sep 28, 2023

Error code 401 #83

Closed

chriskyfung mentioned this issue Nov 18, 2023

Script suddenly stopped working, different error codes #85

Closed

chriskyfung mentioned this issue Jan 1, 2024

Error: Address unavailable (code: 0xf1) #86

Open

chriskyfung closed this as completed in 4767204 Feb 5, 2024

chriskyfung reopened this Feb 6, 2024

chriskyfung mentioned this issue Feb 7, 2024

🐛 Fix #45 Add x-instagram-ajax to request header #59

Merged

IG auto logged me out #45

IG auto logged me out #45

Comments

fifson1 commented Jul 27, 2022 • edited

chriskyfung commented Jul 28, 2022

chriskyfung commented Jul 28, 2022

fifson1 commented Jul 28, 2022

fifson1 commented Jul 28, 2022

chriskyfung commented Jul 29, 2022

fifson1 commented Jul 29, 2022 • edited

fifson1 commented Aug 7, 2022

fifson1 commented Aug 10, 2022

chriskyfung commented Aug 10, 2022

fifson1 commented Aug 10, 2022 • edited

fifson1 commented Aug 13, 2022

fifson1 commented Aug 26, 2022

chriskyfung commented Aug 26, 2022

fifson1 commented Aug 26, 2022

fifson1 commented Sep 17, 2022

chriskyfung commented Sep 18, 2022

chriskyfung commented Oct 11, 2022

fifson1 commented Oct 12, 2022

Seeker220 commented Oct 12, 2022 • edited

chriskyfung commented Oct 12, 2022

Seeker220 commented Oct 12, 2022

chriskyfung commented Oct 12, 2022

Seeker220 commented Oct 12, 2022

Seeker220 commented Oct 12, 2022 • edited

chriskyfung commented Oct 13, 2022

chriskyfung commented Oct 13, 2022 • edited

Seeker220 commented Oct 13, 2022

Seeker220 commented Oct 13, 2022

fifson1 commented Oct 17, 2022 • edited

Seeker220 commented Oct 17, 2022

fifson1 commented Oct 20, 2022 • edited

chriskyfung commented Oct 22, 2022

fifson1 commented Oct 22, 2022

gatoradez commented Mar 21, 2023

chriskyfung commented Mar 24, 2023

Sylphystia commented Aug 24, 2023

chriskyfung commented Feb 4, 2024

Updates

fifson1 commented Jul 27, 2022 •

edited

fifson1 commented Jul 29, 2022 •

edited

fifson1 commented Aug 10, 2022 •

edited

Seeker220 commented Oct 12, 2022 •

edited

Seeker220 commented Oct 12, 2022 •

edited

chriskyfung commented Oct 13, 2022 •

edited

fifson1 commented Oct 17, 2022 •

edited

fifson1 commented Oct 20, 2022 •

edited