![]()
The Topics Extraction API allows you to find key topics within large amounts of text. The extracted topics can be used in a variety of applications such as to create navigable word clouds, characterize documents, features for machine learning, summarizing documents and for visualization of large amounts of text. Topunstr ics can be extracted from single documents or aggregated texts. Our algorithm uses data mining and NLP techniques over graphs.
Key Features
With our API, as output you get a main topic with a corresponding score. You also see related topics if present and topics are single words or phrases. For each topic you are also presented with supporting texts that discusses the topic. Our API is currently fine-tuned for the English language.
Types of Supported Texts
This API will work for most texts in its plain form. Basic pre-processing may be needed for highly noisy texts. If you have questions about how much pre-processing may be needed for your data, you can ask a question at any time. Here is an idea of the types of texts that can be used with the API:
- Unstructured survey responses
- Aggregated news articles
- User reviews
- Blog posts
- Web pages
- Customer complaints
Getting Started with Topics Extraction
API Key
Before you start, please ensure that you have a valid API key to be able to access the API.
Request & Response
The Topics Extraction API accepts a JSON request and returns a JSON response.
Topics Extraction API Request
The topics extraction endpoint accepts two parameters, type and text.
| Parameter Name | Type | Required? | Values |
|---|---|---|---|
| type | text | yes | “chunk” or “pre-sentenced” |
| text | text | yes | a chunk of text or pre-sentenced text (array of sentences) |
“chunk” or “pre-sentenced”, which to use?
When you use “chunk” as the type it simply means you are leaving the topics endpoint to determine sentence boundaries for the text that you send in via the text parameter. We use our default sentence segmentation tool to extract sentences from text. This typically works well for well-written texts such as news articles and legal documents. If you are sending in concatenation of texts as chunk (e.g. concatenated news articles, concatenated Tweets, etc), please ensure that punctuation is available between the two textual units (a “.” should be sufficient).
For inherently short texts such as Tweets, the “pre-sentenced” option would be more appropriate. For the “pre-sentenced” type, you will have to send in an array of texts under the “text” parameter. An example can be found here.
Topics Extraction API Response
The topics extraction API returns several values as the output. The topics are ranked based on its score. The score is essentially computed as a function of the topic length and its importance to the text fed into the API.
| Parameter Name | Type | Description |
|---|---|---|
| mainTopic | text | The primary topic extracted |
| relatedTopics | text | A list of secondary related topics if present |
| score | double | The topic score which is a function of topic length and importance of the topic |
| supportingTexts | text | List of snippets from the original text that discusses the topic |
Example JSON Request
The example below is a news article sent in as a “chunk” of text.
|
1 2 |
{ "type": "chunk", "text": "PARIS (Reuters) - Charlie Hebdo's first edition since an attack by Islamist gunmen sold out within minutes on Wednesday, featuring a cartoon of the Prophet Mohammad on a cover that defenders praised as art but critics saw as a new provocation. French readers queued at dawn for copies to support the satirical newspaper, even as al Qaeda's branch in Yemen claimed responsibility for the attack last week, saying it ordered the killings because it deemed the weekly had insulted the Prophet. Across the Middle East, Muslim leaders who have denounced the attack in which 12 people died called for calm, while criticizing Charlie Hebdo's decision to publish a fresh caricature of Mohammad. President Francois Hollande visited France's Charles de Gaulle aircraft carrier in the Mediterranean and said it was ready to support military operations against Islamic State in Iraq in close cooperation with coalition forces . The U.S. State Department said Secretary of State John Kerry would meet with Hollande in Paris on Friday to offer assistance to France. Also on Wednesday, the Interior Ministry said over 50 cases of people voicing support for terrorism had been registered since the attack on Charlie Hebdo's Paris office and the subsequent killings of a policewoman and four people at a Jewish supermarket. Millions of copies of the survivors' edition were printed, dwarfing the usual 60,000 print run. On its cover, a tearful Mohammad holds a Je suis Charlie sign under the words All is forgiven. View galleryPeople queue to get a copy of satirical French magazine … People queue to get a copy of satirical French magazine Charlie Hebdo new issue titled Tout es … David Sullo, standing at the end of a queue of two dozen people at a central Paris kiosk, said he had never bought it before. It's not quite my political stripes, but it's important for me to buy it today and support freedom of expression, he said. Inside, one cartoon showed jihadists saying, We shouldn't touch Charlie people ... otherwise they will look like martyrs and, once in heaven, these bastards will steal our virgins. This week's edition underlined the irony of how the victims had been commemorated at Notre-Dame cathedral in Paris. What makes us laugh most is that the bells of Notre-Dame rang in our honor, read an editorial in the newspaper, which emerged from the 1968 counter-culture movement and has long mocked all religions and pillars of the establishment. Prime Minister Manuel Valls, himself a frequent target of the weekly's caricatures, left a cabinet meeting with a copy tucked under his arm. BATTLE OF PARIS In a video posted on YouTube, Al Qaeda's Yemen branch said its leadership had ordered last Wednesday's attack on Charlie Hebdo. View galleryA handwritten sign is displayed at a newsstand after … A handwritten sign, which reads No more Charlie , is displayed after it sold out the limi … As for the blessed Battle of Paris, we, the Organization of al Qaeda al Jihad in the Arabian Peninsula, claim responsibility for this operation as vengeance for the Messenger of God, said Nasser bin Ali al-Ansi, the group's main ideologue in Yemen. Ansi said the strike was the implementation of an order by al Qaeda leader Ayman al-Zawahri. It was not immediately possible to verify the authenticity of the recording. The two brothers who attacked Charlie Hebdo and a third gunman who killed the policewoman and hostages at the kosher supermarket all died in police raids. Defenders praised the cover for upholding the newspaper's satirical mission, proclaiming its right to free speech while maintaining a mournful tone and a peaceful message. Jonathan Jones, art critic for Britain's Guardian newspaper, called the cover a life-affirming work of art . Funny people were killed for being funny. This new cover is the only possible response - a response that makes you laugh, he wrote. Belgium's Le Soir wrote, Not publishing the edition would have been like a second death for the victims. View galleryA handwritten sign at a newsstand is displayed after … A handwritten sign at a newsstand, which reads No more Charlie Hebdo , is displayed after … Several German newspapers reprinted the cover. It filled the back page of the top-selling Bild daily, whose columnist Franz Josef Wagner praised it highly. It is sarcasm, it is biting ridicule ... they are mocking the murderers, he wrote. In the Middle East, it was branded a new provocation that could create a backlash. Publishing the cartoons shows contempt for Muslim feelings, said the Grand Mufti of Jerusalem and Palestinian lands, Mohammed Hussein. Iran's foreign minister Mohammad Jawad Zarif said serious dialogue with the West would be easier if it respected Muslim sensitivities. We believe that sanctities need to be respected, he said before nuclear talks in Geneva with U.S. Secretary of State John Kerry. We won't be able to engage in a serious dialogue if we start disrespecting each other's values. A court in the southeastern Turkish city of Diyarbakir ruled in favor of blocking four websites that ran the cover. Those crass people proud of attacking religions are inflaming Islamophobia. This mentality, whatever vehicle it uses, is a threat to world peace, Deputy Prime Minister Yalcin Akdogan wrote on Twitter. Algeria's independent Arab language daily Echorouk ran its own front page cartoon showing a man carrying a Je suis Charlie placard next to an army tank crushing placards from Palestine, Mali, Gaza, Iraq and Syria. Above, the headline reads: We are all Mohammad . Samir Mahmoud, a retired engineer in Cairo, said the cartoons have no meaning, they should not affect us. We as Muslims are bigger and stronger than some cartoon . Days after Senegal's President Macky Sall took part in the Paris march, his interior ministry issued a statement on Wednesday banning the distribution and circulation by all means of the latest edition of Charlie Hebdo and the French daily Liberation . All proceeds from the sale of this week's edition will go directly to Charlie Hebdo, a windfall for a publication that had been struggling financially. While many French people enthusiastically supported the weekly's decision to put another cartoon of Mohammad on its cover, some expressed concern that it would provoke more tension. Bordeaux mosque rector Tareq Oubrou urged French Muslims not to overreact to the new cartoons. I don't think the prophet of Islam needs stupid or excited reactions, he told BFM-TV. Freedom has its downsides and we must live with them. French prosecutors decided to try Dieudonne M'bala M'bala, a French comedian convicted in the past for anti-Semitic comments, on charges of glorifying terrorism for writing on Facebook, Je suis Charlie Coulibaly , adding the surname of one of the gunmen. France will announce next week a set of anti-terrorism measures, officials said. New resources will be released for surveillance and France could look at widening the policy of isolating radical prisoners. Hollande, the most unpopular French president in survey history over his failure to kickstart the economy, has come out of the crisis well, with 85 percent of French approving of his handling of it, a poll showed on Wednesday. (Additional reporting by Emmanuel Jarry, Valerie Parent and Brian Love in Paris, Sami Aboudi in Dubai, Madeleine Chambers in Berlin, Seyhmus Cakan in Diyarbakir and Reuters correspondents in Europe and the Middle East and Africa; Writing by Ingrid Melander and Tom Heneghan; Editing by David Stamp, Toni Reinhold)" } |
Example JSON Response
The response example below shows a list of topics formed for the news article from the request above. Notice that the top topics are related to “charlie hebdo” and the “je suis charlie” sign.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 |
{ "results": [ { "mainTopic": "charlie hebdo:5.35", "relatedTopics": [ "hebdo:1.86", "charlie:1.71" ], "score": "5.35", "supportingTexts": [ "0000:PARIS ( reuters )-charlie hebdo's first edition since an attack by islamist gunmen sold out within minutes on wednesday , featuring a cartoon of the prophet mohammad on a cover that defenders praised as art but critics saw as a new provocation .", "0002:across the middle east , muslim leaders who have denounced the attack in which 12 people died called for calm , while criticizing charlie hebdo's decision to publish a fresh caricature of mohammad .", "0005:also on wednesday , the interior ministry said over 50 cases of people voicing support for terrorism had been registered since the attack on charlie hebdo's paris office and the subsequent killings of a policewoman and four people at a jewish supermarket .", "0007:on its cover , a tearful mohammad holds a je suis charlie sign under the words all is forgiven .", "0008:view gallerypeople queue to get a copy of satirical french magazine & nbsp ; & hellip ; people queue to get a copy of satirical french magazine charlie hebdo new issue titled tout es … david sullo , standing at the end of a queue of two dozen people at a central paris kiosk , said he had never bought it before .", "0010:inside , one cartoon showed jihadists saying , we shouldn't touch charlie people .", "0015:BATTLE OF PARIS in a video posted on youtube , al qaeda's yemen branch said its leadership had ordered last wednesday's attack on charlie hebdo .", "0016:view gallerya handwritten sign is displayed at a newsstand after & nbsp ; & hellip ; a handwritten sign , which reads no more charlie , is displayed after it sold out the limi … as for the blessed battle of paris , we , the organization of al qaeda al jihad in the arabian peninsula , claim responsibility for this operation as vengeance for the messenger of god , said nasser bin ali al-ansi , the group's main ideologue in yemen .", "0019:the two brothers who attacked charlie hebdo and a third gunman who killed the policewoman and hostages at the kosher supermarket all died in police raids .", "0025:view gallerya handwritten sign at a newsstand is displayed after & nbsp ; & hellip ; a handwritten sign at a newsstand , which reads no more charlie hebdo , is displayed after … several german newspapers reprinted the cover .", "0037:algeria's independent arab language daily echorouk ran its own front page cartoon showing a man carrying a je suis charlie placard next to an army tank crushing placards from palestine , mali , gaza , iraq and syria .", "0042:days after senegal's president macky sall took part in the paris march , his interior ministry issued a statement on wednesday banning the distribution and circulation by all means of the latest edition of charlie hebdo and the french daily liberation .", "0043:all proceeds from the sale of this week's edition will go directly to charlie hebdo , a windfall for a publication that had been struggling financially .", "0047:french prosecutors decided to try dieudonne m'bala m'bala , a french comedian convicted in the past for anti-semitic comments , on charges of glorifying terrorism for writing on facebook , je suis charlie coulibaly , adding the surname of one of the gunmen ." ] }, { "mainTopic": "je suis charlie:3.35", "relatedTopics": [ "suis charlie:2.65" ], "score": "3.35", "supportingTexts": [ "0007:on its cover , a tearful mohammad holds a je suis charlie sign under the words all is forgiven .", "0037:algeria's independent arab language daily echorouk ran its own front page cartoon showing a man carrying a je suis charlie placard next to an army tank crushing placards from palestine , mali , gaza , iraq and syria .", "0047:french prosecutors decided to try dieudonne m'bala m'bala , a french comedian convicted in the past for anti-semitic comments , on charges of glorifying terrorism for writing on facebook , je suis charlie coulibaly , adding the surname of one of the gunmen ." ] }, { "mainTopic": "hellip a handwritten sign:3.11", "relatedTopics": [ "sign:2.08" ], "score": "3.11", "supportingTexts": [ "0007:on its cover , a tearful mohammad holds a je suis charlie sign under the words all is forgiven .", "0016:view gallerya handwritten sign is displayed at a newsstand after & nbsp ; & hellip ; a handwritten sign , which reads no more charlie , is displayed after it sold out the limi … as for the blessed battle of paris , we , the organization of al qaeda al jihad in the arabian peninsula , claim responsibility for this operation as vengeance for the messenger of god , said nasser bin ali al-ansi , the group's main ideologue in yemen .", "0025:view gallerya handwritten sign at a newsstand is displayed after & nbsp ; & hellip ; a handwritten sign at a newsstand , which reads no more charlie hebdo , is displayed after … several german newspapers reprinted the cover ." ] }, { "mainTopic": "handwritten sign:3.08", "relatedTopics": [], "score": "3.08", "supportingTexts": [ "0016:view gallerya handwritten sign is displayed at a newsstand after & nbsp ; & hellip ; a handwritten sign , which reads no more charlie , is displayed after it sold out the limi … as for the blessed battle of paris , we , the organization of al qaeda al jihad in the arabian peninsula , claim responsibility for this operation as vengeance for the messenger of god , said nasser bin ali al-ansi , the group's main ideologue in yemen .", "0025:view gallerya handwritten sign at a newsstand is displayed after & nbsp ; & hellip ; a handwritten sign at a newsstand , which reads no more charlie hebdo , is displayed after … several german newspapers reprinted the cover ." ] }, { "mainTopic": "US state john kerry:2.91", "relatedTopics": [ "kerry:2.47", "john kerry:2.23" ], "score": "2.91", "supportingTexts": [ "0004:the US state department said secretary of state john kerry would meet with hollande in paris on friday to offer assistance to france .", "0032:we believe that sanctities need to be respected , he said before nuclear talks in geneva with US secretary of state john kerry ." ] }, { "mainTopic": "middle east:2.65", "relatedTopics": [ "east:2.29" ], "score": "2.65", "supportingTexts": [ "0002:across the middle east , muslim leaders who have denounced the attack in which 12 people died called for calm , while criticizing charlie hebdo's decision to publish a fresh caricature of mohammad .", "0029:in the middle east , it was branded a new provocation that could create a backlash .", "0051:( additional reporting by emmanuel jarry , valerie parent and brian love in paris , sami aboudi in dubai , madeleine chambers in berlin , seyhmus cakan in diyarbakir and reuters correspondents in europe and the middle east and africa ; writing by ingrid melander and tom heneghan ; editing by david stamp , toni reinhold ) " ] }, { "mainTopic": "provocation:2.47", "relatedTopics": [ "new provocation:2.00" ], "score": "2.47", "supportingTexts": [ "0000:PARIS ( reuters )-charlie hebdo's first edition since an attack by islamist gunmen sold out within minutes on wednesday , featuring a cartoon of the prophet mohammad on a cover that defenders praised as art but critics saw as a new provocation .", "0029:in the middle east , it was branded a new provocation that could create a backlash ." ] }, { "mainTopic": "victims:2.47", "relatedTopics": [], "score": "2.47", "supportingTexts": [ "0012:this week's edition underlined the irony of how the victims had been commemorated at notre-dame cathedral in paris .", "0024:belgium's le soir wrote , not publishing the edition would have been like a second death for the victims ." ] }, { "mainTopic": "laugh:2.47", "relatedTopics": [], "score": "2.47", "supportingTexts": [ "0013:what makes us laugh most is that the bells of notre-dame rang in our honor , read an editorial in the newspaper , which emerged from the 1968 counter-culture movement and has long mocked all religions and pillars of the establishment .", "0023:this new cover is the only possible response-a response that makes you laugh , he wrote ." ] }, { "mainTopic": "gunmen:2.47", "relatedTopics": [], "score": "2.47", "supportingTexts": [ "0000:PARIS ( reuters )-charlie hebdo's first edition since an attack by islamist gunmen sold out within minutes on wednesday , featuring a cartoon of the prophet mohammad on a cover that defenders praised as art but critics saw as a new provocation .", "0047:french prosecutors decided to try dieudonne m'bala m'bala , a french comedian convicted in the past for anti-semitic comments , on charges of glorifying terrorism for writing on facebook , je suis charlie coulibaly , adding the surname of one of the gunmen ." ] }, { "mainTopic": "respected:2.47", "relatedTopics": [], "score": "2.47", "supportingTexts": [ "0031:iran's foreign minister mohammad jawad zarif said serious dialogue with the west would be easier if it respected muslim sensitivities .", "0032:we believe that sanctities need to be respected , he said before nuclear talks in geneva with US secretary of state john kerry ." ] }, { "mainTopic": "ansi:2.47", "relatedTopics": [ "said:1.73" ], "score": "2.47", "supportingTexts": [ "0003:president francois hollande visited france's charles de gaulle aircraft carrier in the mediterranean and said it was ready to support military operations against islamic state in iraq in close cooperation with coalition forces .", "0004:the US state department said secretary of state john kerry would meet with hollande in paris on friday to offer assistance to france .", "0005:also on wednesday , the interior ministry said over 50 cases of people voicing support for terrorism had been registered since the attack on charlie hebdo's paris office and the subsequent killings of a policewoman and four people at a jewish supermarket .", "0008:view gallerypeople queue to get a copy of satirical french magazine & nbsp ; & hellip ; people queue to get a copy of satirical french magazine charlie hebdo new issue titled tout es … david sullo , standing at the end of a queue of two dozen people at a central paris kiosk , said he had never bought it before .", "0009:it's not quite my political stripes , but it's important for me to buy it today and support freedom of expression , he said .", "0015:BATTLE OF PARIS in a video posted on youtube , al qaeda's yemen branch said its leadership had ordered last wednesday's attack on charlie hebdo .", "0016:view gallerya handwritten sign is displayed at a newsstand after & nbsp ; & hellip ; a handwritten sign , which reads no more charlie , is displayed after it sold out the limi … as for the blessed battle of paris , we , the organization of al qaeda al jihad in the arabian peninsula , claim responsibility for this operation as vengeance for the messenger of god , said nasser bin ali al-ansi , the group's main ideologue in yemen .", "0017:ansi said the strike was the implementation of an order by al qaeda leader ayman al-zawahri .", "0030:publishing the cartoons shows contempt for muslim feelings , said the grand mufti of jerusalem and palestinian lands , mohammed hussein .", "0031:iran's foreign minister mohammad jawad zarif said serious dialogue with the west would be easier if it respected muslim sensitivities .", "0032:we believe that sanctities need to be respected , he said before nuclear talks in geneva with US secretary of state john kerry .", "0040:samir mahmoud , a retired engineer in cairo , said the cartoons have no meaning , they should not affect us .", "0048:france will announce next week a set of anti-terrorism measures , officials said ." ] }, { "mainTopic": "saying:2.47", "relatedTopics": [], "score": "2.47", "supportingTexts": [ "0001:french readers queued at dawn for copies to support the satirical newspaper , even as al qaeda's branch in yemen claimed responsibility for the attack last week , saying it ordered the killings because it deemed the weekly had insulted the prophet .", "0010:inside , one cartoon showed jihadists saying , we shouldn't touch charlie people ." ] }, { "mainTopic": "supermarket:2.47", "relatedTopics": [], "score": "2.47", "supportingTexts": [ "0005:also on wednesday , the interior ministry said over 50 cases of people voicing support for terrorism had been registered since the attack on charlie hebdo's paris office and the subsequent killings of a policewoman and four people at a jewish supermarket .", "0019:the two brothers who attacked charlie hebdo and a third gunman who killed the policewoman and hostages at the kosher supermarket all died in police raids ." ] }, { "mainTopic": "reads:2.29", "relatedTopics": [], "score": "2.29", "supportingTexts": [ "0016:view gallerya handwritten sign is displayed at a newsstand after & nbsp ; & hellip ; a handwritten sign , which reads no more charlie , is displayed after it sold out the limi … as for the blessed battle of paris , we , the organization of al qaeda al jihad in the arabian peninsula , claim responsibility for this operation as vengeance for the messenger of god , said nasser bin ali al-ansi , the group's main ideologue in yemen .", "0025:view gallerya handwritten sign at a newsstand is displayed after & nbsp ; & hellip ; a handwritten sign at a newsstand , which reads no more charlie hebdo , is displayed after … several german newspapers reprinted the cover .", "0038:above , the headline reads ." ] }, { "mainTopic": "daily:2.29", "relatedTopics": [], "score": "2.29", "supportingTexts": [ "0026:it filled the back page of the top-selling bild daily , whose columnist franz josef wagner praised it highly .", "0037:algeria's independent arab language daily echorouk ran its own front page cartoon showing a man carrying a je suis charlie placard next to an army tank crushing placards from palestine , mali , gaza , iraq and syria .", "0042:days after senegal's president macky sall took part in the paris march , his interior ministry issued a statement on wednesday banning the distribution and circulation by all means of the latest edition of charlie hebdo and the french daily liberation ." ] }, { "mainTopic": "newsstand:2.29", "relatedTopics": [], "score": "2.29", "supportingTexts": [ "0016:view gallerya handwritten sign is displayed at a newsstand after & nbsp ; & hellip ; a handwritten sign , which reads no more charlie , is displayed after it sold out the limi … as for the blessed battle of paris , we , the organization of al qaeda al jihad in the arabian peninsula , claim responsibility for this operation as vengeance for the messenger of god , said nasser bin ali al-ansi , the group's main ideologue in yemen .", "0025:view gallerya handwritten sign at a newsstand is displayed after & nbsp ; & hellip ; a handwritten sign at a newsstand , which reads no more charlie hebdo , is displayed after … several german newspapers reprinted the cover ." ] }, { "mainTopic": "art:2.29", "relatedTopics": [], "score": "2.29", "supportingTexts": [ "0000:PARIS ( reuters )-charlie hebdo's first edition since an attack by islamist gunmen sold out within minutes on wednesday , featuring a cartoon of the prophet mohammad on a cover that defenders praised as art but critics saw as a new provocation .", "0021:jonathan jones , art critic for britain's guardian newspaper , called the cover a life-affirming work of art ." ] }, { "mainTopic": "hollande:2.29", "relatedTopics": [], "score": "2.29", "supportingTexts": [ "0003:president francois hollande visited france's charles de gaulle aircraft carrier in the mediterranean and said it was ready to support military operations against islamic state in iraq in close cooperation with coalition forces .", "0004:the US state department said secretary of state john kerry would meet with hollande in paris on friday to offer assistance to france .", "0050:hollande , the most unpopular french president in survey history over his failure to kickstart the economy , has come out of the crisis well , with 85 percent of french approving of his handling of it , a poll showed on wednesday ." ] }, { "mainTopic": "cartoons:2.29", "relatedTopics": [ "cartoon:2.08" ], "score": "2.29", "supportingTexts": [ "0000:PARIS ( reuters )-charlie hebdo's first edition since an attack by islamist gunmen sold out within minutes on wednesday , featuring a cartoon of the prophet mohammad on a cover that defenders praised as art but critics saw as a new provocation .", "0010:inside , one cartoon showed jihadists saying , we shouldn't touch charlie people .", "0030:publishing the cartoons shows contempt for muslim feelings , said the grand mufti of jerusalem and palestinian lands , mohammed hussein .", "0037:algeria's independent arab language daily echorouk ran its own front page cartoon showing a man carrying a je suis charlie placard next to an army tank crushing placards from palestine , mali , gaza , iraq and syria .", "0040:samir mahmoud , a retired engineer in cairo , said the cartoons have no meaning , they should not affect us .", "0041:we as muslims are bigger and stronger than some cartoon .", "0044:while many french people enthusiastically supported the weekly's decision to put another cartoon of mohammad on its cover , some expressed concern that it would provoke more tension .", "0045:bordeaux mosque rector tareq oubrou urged french muslims not to overreact to the new cartoons ." ] }, { "mainTopic": "yemen:2.29", "relatedTopics": [], "score": "2.29", "supportingTexts": [ "0001:french readers queued at dawn for copies to support the satirical newspaper , even as al qaeda's branch in yemen claimed responsibility for the attack last week , saying it ordered the killings because it deemed the weekly had insulted the prophet .", "0015:BATTLE OF PARIS in a video posted on youtube , al qaeda's yemen branch said its leadership had ordered last wednesday's attack on charlie hebdo .", "0016:view gallerya handwritten sign is displayed at a newsstand after & nbsp ; & hellip ; a handwritten sign , which reads no more charlie , is displayed after it sold out the limi … as for the blessed battle of paris , we , the organization of al qaeda al jihad in the arabian peninsula , claim responsibility for this operation as vengeance for the messenger of god , said nasser bin ali al-ansi , the group's main ideologue in yemen ." ] }, { "mainTopic": "prophet:2.29", "relatedTopics": [], "score": "2.29", "supportingTexts": [ "0000:PARIS ( reuters )-charlie hebdo's first edition since an attack by islamist gunmen sold out within minutes on wednesday , featuring a cartoon of the prophet mohammad on a cover that defenders praised as art but critics saw as a new provocation .", "0001:french readers queued at dawn for copies to support the satirical newspaper , even as al qaeda's branch in yemen claimed responsibility for the attack last week , saying it ordered the killings because it deemed the weekly had insulted the prophet .", "0046:i don't think the prophet of islam needs stupid or excited reactions , he told BFM-TV freedom has its downsides and we must live with them ." ] }, { "mainTopic": "week:2.17", "relatedTopics": [], "score": "2.17", "supportingTexts": [ "0001:french readers queued at dawn for copies to support the satirical newspaper , even as al qaeda's branch in yemen claimed responsibility for the attack last week , saying it ordered the killings because it deemed the weekly had insulted the prophet .", "0012:this week's edition underlined the irony of how the victims had been commemorated at notre-dame cathedral in paris .", "0043:all proceeds from the sale of this week's edition will go directly to charlie hebdo , a windfall for a publication that had been struggling financially .", "0048:france will announce next week a set of anti-terrorism measures , officials said ." ] }, { "mainTopic": "newspaper:2.17", "relatedTopics": [], "score": "2.17", "supportingTexts": [ "0001:french readers queued at dawn for copies to support the satirical newspaper , even as al qaeda's branch in yemen claimed responsibility for the attack last week , saying it ordered the killings because it deemed the weekly had insulted the prophet .", "0013:what makes us laugh most is that the bells of notre-dame rang in our honor , read an editorial in the newspaper , which emerged from the 1968 counter-culture movement and has long mocked all religions and pillars of the establishment .", "0020:defenders praised the cover for upholding the newspaper's satirical mission , proclaiming its right to free speech while maintaining a mournful tone and a peaceful message .", "0021:jonathan jones , art critic for britain's guardian newspaper , called the cover a life-affirming work of art ." ] }, { "mainTopic": "france:2.17", "relatedTopics": [], "score": "2.17", "supportingTexts": [ "0003:president francois hollande visited france's charles de gaulle aircraft carrier in the mediterranean and said it was ready to support military operations against islamic state in iraq in close cooperation with coalition forces .", "0004:the US state department said secretary of state john kerry would meet with hollande in paris on friday to offer assistance to france .", "0048:france will announce next week a set of anti-terrorism measures , officials said .", "0049:new resources will be released for surveillance and france could look at widening the policy of isolating radical prisoners ." ] }, { "mainTopic": "wrote:2.17", "relatedTopics": [], "score": "2.17", "supportingTexts": [ "0023:this new cover is the only possible response-a response that makes you laugh , he wrote .", "0024:belgium's le soir wrote , not publishing the edition would have been like a second death for the victims .", "0028:they are mocking the murderers , he wrote .", "0036:this mentality , whatever vehicle it uses , is a threat to world peace , deputy prime minister yalcin akdogan wrote on twitter ." ] }, { "mainTopic": "wednesday:2.08", "relatedTopics": [], "score": "2.08", "supportingTexts": [ "0000:PARIS ( reuters )-charlie hebdo's first edition since an attack by islamist gunmen sold out within minutes on wednesday , featuring a cartoon of the prophet mohammad on a cover that defenders praised as art but critics saw as a new provocation .", "0005:also on wednesday , the interior ministry said over 50 cases of people voicing support for terrorism had been registered since the attack on charlie hebdo's paris office and the subsequent killings of a policewoman and four people at a jewish supermarket .", "0015:BATTLE OF PARIS in a video posted on youtube , al qaeda's yemen branch said its leadership had ordered last wednesday's attack on charlie hebdo .", "0042:days after senegal's president macky sall took part in the paris march , his interior ministry issued a statement on wednesday banning the distribution and circulation by all means of the latest edition of charlie hebdo and the french daily liberation .", "0050:hollande , the most unpopular french president in survey history over his failure to kickstart the economy , has come out of the crisis well , with 85 percent of french approving of his handling of it , a poll showed on wednesday ." ] }, { "mainTopic": "mohammad:2.01", "relatedTopics": [], "score": "2.01", "supportingTexts": [ "0000:PARIS ( reuters )-charlie hebdo's first edition since an attack by islamist gunmen sold out within minutes on wednesday , featuring a cartoon of the prophet mohammad on a cover that defenders praised as art but critics saw as a new provocation .", "0002:across the middle east , muslim leaders who have denounced the attack in which 12 people died called for calm , while criticizing charlie hebdo's decision to publish a fresh caricature of mohammad .", "0007:on its cover , a tearful mohammad holds a je suis charlie sign under the words all is forgiven .", "0031:iran's foreign minister mohammad jawad zarif said serious dialogue with the west would be easier if it respected muslim sensitivities .", "0039:we are all mohammad .", "0044:while many french people enthusiastically supported the weekly's decision to put another cartoon of mohammad on its cover , some expressed concern that it would provoke more tension ." ] }, { "mainTopic": "paris:1.95", "relatedTopics": [], "score": "1.95", "supportingTexts": [ "0004:the US state department said secretary of state john kerry would meet with hollande in paris on friday to offer assistance to france .", "0005:also on wednesday , the interior ministry said over 50 cases of people voicing support for terrorism had been registered since the attack on charlie hebdo's paris office and the subsequent killings of a policewoman and four people at a jewish supermarket .", "0008:view gallerypeople queue to get a copy of satirical french magazine & nbsp ; & hellip ; people queue to get a copy of satirical french magazine charlie hebdo new issue titled tout es … david sullo , standing at the end of a queue of two dozen people at a central paris kiosk , said he had never bought it before .", "0012:this week's edition underlined the irony of how the victims had been commemorated at notre-dame cathedral in paris .", "0016:view gallerya handwritten sign is displayed at a newsstand after & nbsp ; & hellip ; a handwritten sign , which reads no more charlie , is displayed after it sold out the limi … as for the blessed battle of paris , we , the organization of al qaeda al jihad in the arabian peninsula , claim responsibility for this operation as vengeance for the messenger of god , said nasser bin ali al-ansi , the group's main ideologue in yemen .", "0042:days after senegal's president macky sall took part in the paris march , his interior ministry issued a statement on wednesday banning the distribution and circulation by all means of the latest edition of charlie hebdo and the french daily liberation .", "0051:( additional reporting by emmanuel jarry , valerie parent and brian love in paris , sami aboudi in dubai , madeleine chambers in berlin , seyhmus cakan in diyarbakir and reuters correspondents in europe and the middle east and africa ; writing by ingrid melander and tom heneghan ; editing by david stamp , toni reinhold ) " ] }, { "mainTopic": "cover:1.90", "relatedTopics": [], "score": "1.90", "supportingTexts": [ "0000:PARIS ( reuters )-charlie hebdo's first edition since an attack by islamist gunmen sold out within minutes on wednesday , featuring a cartoon of the prophet mohammad on a cover that defenders praised as art but critics saw as a new provocation .", "0007:on its cover , a tearful mohammad holds a je suis charlie sign under the words all is forgiven .", "0020:defenders praised the cover for upholding the newspaper's satirical mission , proclaiming its right to free speech while maintaining a mournful tone and a peaceful message .", "0021:jonathan jones , art critic for britain's guardian newspaper , called the cover a life-affirming work of art .", "0023:this new cover is the only possible response-a response that makes you laugh , he wrote .", "0025:view gallerya handwritten sign at a newsstand is displayed after & nbsp ; & hellip ; a handwritten sign at a newsstand , which reads no more charlie hebdo , is displayed after … several german newspapers reprinted the cover .", "0034:a court in the southeastern turkish city of diyarbakir ruled in favor of blocking four websites that ran the cover .", "0044:while many french people enthusiastically supported the weekly's decision to put another cartoon of mohammad on its cover , some expressed concern that it would provoke more tension ." ] }, { "mainTopic": "people:1.86", "relatedTopics": [], "score": "1.86", "supportingTexts": [ "0002:across the middle east , muslim leaders who have denounced the attack in which 12 people died called for calm , while criticizing charlie hebdo's decision to publish a fresh caricature of mohammad .", "0005:also on wednesday , the interior ministry said over 50 cases of people voicing support for terrorism had been registered since the attack on charlie hebdo's paris office and the subsequent killings of a policewoman and four people at a jewish supermarket .", "0008:view gallerypeople queue to get a copy of satirical french magazine & nbsp ; & hellip ; people queue to get a copy of satirical french magazine charlie hebdo new issue titled tout es … david sullo , standing at the end of a queue of two dozen people at a central paris kiosk , said he had never bought it before .", "0010:inside , one cartoon showed jihadists saying , we shouldn't touch charlie people .", "0022:funny people were killed for being funny .", "0035:those crass people proud of attacking religions are inflaming islamophobia .", "0044:while many french people enthusiastically supported the weekly's decision to put another cartoon of mohammad on its cover , some expressed concern that it would provoke more tension ." ] } ] } |
Post-processing
Suppressing Unwanted Topics
In the JSON response above, notice that some of the topics may not be interesting to an application. For example, the topic “saying” may not necessarily be useful. While we have eliminated some non-interesting words as topics, to avoid over eliminating topics, we have let it to the consuming application to further suppress non-interesting topics. This can be done in several ways. First, you can maintain a stop word list customized to your application and remove topics that contain those stop words. Second, you can look at # of supporting texts and score. If a given topic does not have too many supporting texts (e.g. # texts lower than 4) or a score lower than a particular threshold (e.g. score < 2.5) then such topics can be eliminated. You will have to play around to see what works for your application.
Pre-processing
How much pre-processing is needed?
It depends on your text. If you are dealing with fairly noisy texts such as Tweets, you would have to remove retweets, links, hash symbols, unwanted characters and etc. Our API handles basic pre-processing of text including lowercasing, normalization and simple stop word handling.
Language Support
This API is currently only tuned and optimized for English.
