Skip to main content

Social Intelligence Connector

Kayleigh Gaddor
  • Updated

Data connector details

  • Category: Social Media

  • Connector type: API

Description

The Social Intelligence connector allows you to connect with Twitter, Forums, News, Blogs, and Instagram, singly, or all in one shot. 

 

The connector searches through the data word by word, comparing posts with the query, and crawls the post when there is an exact match. It starts crawling the data of the most recent post, and continues for the number of days specified in the schedule.

 

API limitations

Stratifyd can adjust the frequency of data calls to accommodate your needs. Contact your Stratifyd representative to discuss how we may best help you.

 

If you have issues with pulling data, try narrowing your date range on the second page of the wizard or making your query more specific.

 

Input fields

When setting up the connector, provide a search query and select the sources from which to pull the data.

  • Search Query: Enter your search query using the Query rules below.

  • Custom Name: Enter a name to use for the data stream. This is returned in the dc_crawler_name field.

  • Content Limit per month: Set a limit for the number of records to return per month. The default value is

  • Select: Select the sources from which you want to pull data. The options are: Twitter, Instagram, Blogs, Forums, News

You can change your date range on the second page of the wizard, or optionally Schedule data crawls to update your data stream periodically.

 

Data dictionary

This connector returns the following fields that you can use in analysis. 

 

author

  • Description: User name of the content author.

  • Category: user index

  • Example: Shev1996

content

  • Description: Free-form text of the tweet, Instagram post, blog post, forum post, or news article.

  • Category: text index

  • Example: Headline news on the BBC site is automatically world news. You can change this to UK or even England news at a click and this won’t then be included. But this is world news just as the Trump impeachment has been top many times

content_link

  • Description: URL of the source of the content.

  • Category: meta

  • Example: https://www.mumsnet.com/Talk/am_i_being_unreasonable/3806366-To-think-its-odd-that-BBC-news-main-headline?pg=5#93451598

crawler_time_stamp

  • Description: Date and time the data was crawled in numeric format.

  • Category: meta

  • Example: 1580151487848

data_source

  • Description: Name of the data connector used to retrieve the data.

  • Category: meta

  • Example: Social Intelligence

dc_crawler_name

  • Description: Name supplied in the Custom Name field for the data stream.

  • Category: meta

  • Example: Social Intelligence Impeachment UK Opinions

language

  • Description: Two-letter language code of the language detected for the record.

  • Category: meta

  • Example: en

location_country

  • Description: Country of the author of the content.

  • Category: geo index

  • Example: GB

location_info

  • Description: City and state of the content author.

  • Category: geo index

  • Example: , , UK

source

  • Description: Source of the content: twitter, instagram, blog, forum, or news.

  • Category: meta

  • Example: forum

time_stamp

  • Description: Date and time on which the content was created.

  • Category: date index

  • Example: 1580090880000

title

  • Description: Free-form text of any title for the content.

  • Category: text index

  • Example: To think it's odd that BBC news main headline...

Data

  • Description: Returns all of the data fields.

 

Query rules 

For each query, provide at least one search term (keyword or phrase).

 

Regardless of language, surround each search term with English double quotes. (English single-word queries do not require quotation marks.)


e.g. "Jour de la Terre" OR "earth day" OR "地球日" OR "アースデー" OR EarthDay

 

Chinese, Japanese, and Korean languages query by character if you do not wrap search terms (except for country) with ENGLISH double quotation marks. For accurate results, always surround words or phrases with English double quotation marks.

 

For example:

(("古典音乐" OR "嘻哈音乐" OR "流行音乐") AND country:China) AND NOT (country:Taiwan OR city:"Hong Kong")

Boolean operators

Connect multiple search terms with Boolean operators. Search terms are not case sensitive, but capitalizing the operators is a best practice to avoid confusion.

  • AND All of the search terms linked by this operator must exist in the post.e.g. "dog" AND "cat" returns all mentions with both dog and cat.

  • OR At least one of the search terms linked by this operator must exist in the post. e.g. "dog" OR "cat" returns all mentions with either dog or cat.

  • AND NOT None of the search terms following this operator can exist in the post.e.g. "dog" AND "cat" AND NOT "horse" returns all mentions with both dog and cat that do not include horse.

Separate plurals and word stems with the OR operator.

 

e.g. "bulldog" OR "bulldogs" returns  
"bark" OR "barking" OR "barked"

 

You can also return all variations of word stems using a wildcard character, but this character cannot be enclosed in parentheses.

 

e.g. "bark*"

 

Combining operators

Use parentheses to wrap logical phrases so that you can nest queries and set the order in which the operations run.

 

e.g. ("tabby" OR "shorthair" OR "ragdoll") AND ("cat" OR "cats") AND NOT "mix"
("Electronic Music" AND "Pop Music") AND (country:China OR country:USA)
("Electronic Music" AND "Pop Music") AND NOT (country:China)

 

Proximity queries

To search for mentions where the search terms appear within a certain number (n) of words from one another, create a proximity query by adding a tilde symbol and the number of words after the search term (~n). 

e.g. "british shorthair"~3 AND "shorthair" AND NOT "american shorthair"~3

 

Advanced operators

Express advanced operators such as country or city as country:countryName  without quotation marks, unless the value contains multiple words, in which case the multi-word value must be wrapped in quotation marks.

e.g. country:France OR country:China OR country:"New Zealand"

Express advanced operators that take numeric values as postcount:1000 without quotation marks. To express a range of numeric values, use brackets.

e.g. postcount:[50 TO 100]

 

Further questions?

We're here to help! Don't hesitate to contact us for further assistance via chat or submit a ticket!

 

 

 

 

Comments

0 comments

Please sign in to leave a comment.