Miscellaneous ⇒ User & Newbie Resources :: Archives ⇒ Coat Hangers, Duct Tape and auto-posting RSS feeds as News :: Archived ⇒ Community Forums ⇒ CPG Dragonfly™ CMS
Forum IndexUser & Newbie Resources

Archived ⇒ Coat Hangers, Duct Tape and auto-posting RSS feeds as News


I have three main doubts. Would love to see someone help me with these.

1) I happened to visit Albuirs website and he has added the source in every News item. How can I do that to avoid myslef from getting into copyright problems?

2) How do I tell it to break the paragraph and post the rest into the Read More section?

3) Lastly, How can I select the category under which the news item is posted?

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Linux/2.0.54 (Unix)/5.0.51/5.2.6/DragonflyCMS 9.2.1


And one more doubt. I had a look around your website and the System just puches in a few words and then outs the link 'read more here' which links it to the entire page. I intend to paste the whole article. Is it possible?

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Linux/2.0.54 (Unix)/5.0.51/5.2.6/DragonflyCMS 9.2.1


CatchImran wrote
I am a noob when it comes to handling PHP so I need help desperately.

I don't think you will be able to run CaRP within DF without any PHP and MySQL skills. You must be able to modify the code and your DB. At least you have to implement CaRP yourself on your site. For me, it was a motivation to start learning PHP, but I'm still a PHP noob. I don't think I can help you if you get in trouble.
I changed the code of sarah hardly to fit my own needs. What I can offer, is to post it here, if someone is interested in it.

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Linux/Apache/MySQL 5.0.45/PHP 5.2.2/DF 9.2.1


CatchImran wrote
I have three main doubts. Would love to see someone help me with these.

1) I happened to visit Albuirs website and he has added the source in every News item. How can I do that to avoid myslef from getting into copyright problems?

2) How do I tell it to break the paragraph and post the rest into the Read More section?

3) Lastly, How can I select the category under which the news item is posted?


1) The copyright seems to be the biggest problem when using rss feeds. To mention the "source" does not protect you from troubles when using it. As far as I know, everything is protected by the copyright laws, also if you post only a couple of words. If you want to be sure, you have to ask the owner of the feed.
2) You have to code it yourself.
3) You have to change the code. (see the instructions of sarah)

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Linux/Apache/MySQL 5.0.45/PHP 5.2.2/DF 9.2.1


CatchImran wrote
And one more doubt. I had a look around your website and the System just puches in a few words and then outs the link 'read more here' which links it to the entire page. I intend to paste the whole article. Is it possible?

No problem. I just truncated the text. It motivates (hopefully) the author to not complain about copyright. Like this, it's rather publicity then stealing content, because the interested user has to go to the original site to read the whole stuff.

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Linux/Apache/MySQL 5.0.45/PHP 5.2.2/DF 9.2.1


I'm not sure how well this is still working but I have found that several updates have been made to CaRP Evolutions plugin addressing that might cause some modifications. Not sure if Sarah still follows this or not, but I will post my updates shortly.

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Linux/Apache/MySQL/PHP/CPGNuke 9.1.2.1


Well, here is a start from my end, they changed things up in CaRP, so first off, the rss.php file for me needed to look like this:

<?php require_once '$HOME/carpdir/carp.php'; CarpLoadPlugin('mysql.php'); CarpLoadPlugin('replacetext.php'); /*** Setup Code ***/ CarpConf('cache-method','mysql'); MysqlCarpConf('connect',1); MysqlCarpConf('dbname','XXXXXXXXX'); MysqlCarpConf('host','XXXXXXXXX'); MysqlCarpConf('username','XXXXXXXX'); MysqlCarpConf('password','XXXXXXXX'); /*** End Setup Code ***/ // Add any desired configuration settings before CarpCacheShow // using "CarpConf" and other functions CarpConf('poweredby',''); CarpCacheShow('http://feeds.warhammerherald.com/news/'); CarpCacheShow('http://www.warcry.com/rss/news/0.xml'); CarpCacheShow('http://www.rpgwatch.com/files/rss2.xml'); CarpCacheShow('http://images.mmorpg.com/rss/mmorpg_news_rss.xml'); ?>

Next was the feed-loading code which I also changed up a bit to match my usernames, as well as for multiple topics instead of a single-topic.

<?php $dbuser = "databaseusername"; $dbpwd = "databasepass"; $dbname = "databasename"; $dbhost = "databasehost"; mysql_connect($dbhost,$dbuser,$dbpwd); @mysql_select_db($dbname) or die( "Unable to select database"); $query = "update rssitems set descr = concat('Warhammer News from <a href=\"http://www.warherald.net\" rel=\"nofollow\">Warherald</a>!<br /><br />', descr, '<br /><br />Get the full article on Warherald <a href=\"', url, '\" rel=\"nofollow\">here</a>!') where feed_id = '1' and news_submitted = '0'"; $result = $command; mysql_query($query); $query2 = "update rssitems set descr = concat('News from <a href=\"http://www.rpgwatch.com\" rel=\"nofollow\">RPGWatch</a>!<br /><br />', descr, '<br /><br />Get the rest at RPGWatch <a href=\"', url, '\" rel=\"nofollow\">here</a>!') where feed_id = '3' and news_submitted = '0'"; $result = $command; mysql_query($query2); $query3 = "update rssitems set descr = concat('Updates from <a href=\"http://www.warcry.com\" rel=\"nofollow\">Warcry</a>!<br /><br />', descr, '<br /><br />See the full story at Warcry <a href=\"', url, '\" rel=\"nofollow\">here</a>!') where feed_id = '2' and news_submitted = '0'"; $result = $command; mysql_query($query3); $query4 = "update rssitems set descr = concat('Game news from <a href=\"http://www.mmorpg.com\" rel=\"nofollow\">MMORPG.com</a>!<br /><br />', descr, '<br /><br />Read more on this article from MMORPG <a href=\"', url, '\" rel=\"nofollow\">here</a>!') where feed_id = '4' and news_submitted = '0'"; $result = $command; mysql_query($query4); $query5 = "insert into df_stories (title, informant, hometext, bodytext, time, topic, ihome) select title, 'USER', title, descr, posted, '5', '1' from rssitems where news_submitted = '0' and feed_id = '1';"; $result = $command; mysql_query($query5); $query6 = "insert into df_stories (title, informant, hometext, bodytext, time, topic, ihome) select title, 'USER', title, descr, posted, '6', '1' from rssitems where news_submitted = '0' and feed_id = '3';"; $result = $command; mysql_query($query6); $query7 = "insert into df_stories (title, informant, hometext, bodytext, time, topic, ihome) select title, 'USER', title, descr, posted, '6', '1' from rssitems where news_submitted = '0' and feed_id = '2';"; $result = $command; mysql_query($query7); $query8 = "insert into df_stories (title, informant, hometext, bodytext, time, topic, ihome) select title, 'USER', title, descr, posted, '6', '1' from rssitems where news_submitted = '0' and feed_id = '4';"; $result = $command; mysql_query($query8); $query9 = "update rssitems set news_submitted = '1'"; $result = $command; mysql_query($query9); mysql_close() ?>

To note, I added title twice because DF will make a blank topic when looking at the archives and it looks icky, and also set the "ihome" (in home) variable to 1 so things automatically displayed on the front page for me.

I would really like to commission some assistance in furthering the topic classification portion of things. I would particularly be interested in re-assigning topics based on keyword content however don't know exactly where to begin on that portion.

That's all I have for now, oh, and make sure to place the appropriate DB connection info into carpconf.php as well after you setup CaRP. It reminds you of this, but sometimes one has to {expletive removed} twice to get it....

For anyone who would like to help, or take me up on my offer for topic-classification please PM me, I'm sure the effort is very minimal.

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Linux/Apache/MySQL/PHP/CPGNuke 9.1.2.1


Well I changed a lot in Sarahs original code. Here's what I use at the moment:

I use two files because it makes it much easier to manage the feeds, if several feeds are used:
-> rss.php: Configruration of the feeds (I've listed two examples 1st is one for normal rss feeds, the 2nd is one for Amazon feeds. Just copy the whole code block as much times as you like)
-> rss_update.inc: processes the changes: trims the whitspace, update the string replacings, adds the source and author lines to the content, adds images, avoid duplicates, and moves the news into the cms_queue or cms_stories table.

rss.php
<?php ////////////////////////////////// //// START CaRP CONFIGURATION //// ////////////////////////////////// require_once '/home/www/myaccount/xxxmy_path_to_carp/carp/carp.php'; /*** Setup Code ***/ CarpConf('cache-method','mysql'); CarpConf('mysql-connect',1); CarpConf('mysql-database','xxxxx'); CarpConf('mysql-username','xxxxx'); CarpConf('mysql-password','xxxxx'); /*** End Setup Code ***/ /*** Load Plugins ***/ CarpLoadPlugin('mysql.php'); /*** End Load Plugins ***/ // Add any desired configuration settings before CarpCacheShow // using "CarpConf" and other functions /*** CarpConf ***/ //CarpConf('encodingin','ISO-8859-1'); CarpConf('encodingout','UTF-8'); CarpConf('corder',''); CarpConf('iorder','image,link,author,desc'); CarpConf('cacheinterval',720); CarpConf('maxitems',15); CarpConf('linktarget',1); CarpConf('aidesc',''); CarpConf('ai','<br clear="all" /> <br />'); CarpConf('biimage','<div style="float:left;margin-right:5px;">'); CarpConf('aiimage','</div>'); /*** End CarpConf ***/ /*** RSS Feeds to get ***/ // 1st Feed CarpCacheShow('http://www.myfeedurl.com/xxx'); // 2nd Feed: Amazon US $q='Enter_your_Amazon_search_term_here'; //$MaxItems=3; $OneReview=1; $MaxReviewLength=400; $SearchIndex='Books'; $xslt='http://www.mydomain.com/xxx/Amazon.xsl'; $AccessKeyId='xxxxx'; $AssociateTag='xxxxx'; $country='US'; $countries=array( 'US'=>'xml-us.amznxslt.com', 'DE'=>'xml-de.amznxslt.com', 'FR'=>'xml-fr.amznxslt.com', 'CA'=>'xml-ca.amznxslt.com', 'JP'=>'xml-jp.amznxslt.com', 'UK'=>'xml-uk.amznxslt.com' ); if (!($MaxItems+=0)) $MaxItems=10; if (!($OneReview+=0)) $OneReview=1; $MaxReviewLength+=0; if (!isset($SearchIndex)) $SearchIndex='Books'; $amazonquery='http://'.$countries[$country].'/onca/xml?Service=AWSECommerceService'. '&Operation=ItemSearch&ResponseGroup=Small,Reviews,Images,Offers'. "&AWSAccessKeyId=$AccessKeyId&AssociateTag=$AssociateTag". "&Sort=salesrank&Style=".urlencode($xslt)."&Keywords=".urlencode($q). "&SearchIndex=".urlencode($SearchIndex). ($OneReview?'&CaRPOneReview=1':''). ($MaxReviewLength?"&CaRPMaxReviewLength=$MaxReviewLength":''); if (1) echo "<p>$amazonquery</p>"; CarpCacheShow($amazonquery); ///////////////////////////////// //// STOP CaRP CONFIGURATION //// ///////////////////////////////// ////////////////////////////// //// START PROCESS SCRIPT //// ////////////////////////////// //DB configruation for private script $dbuser = 'xxxxx'; $dbpwd = 'xxxxx'; $dbname = 'xxxxx'; $dbhost = 'localhost'; mysql_connect($dbhost,$dbuser,$dbpwd); @mysql_select_db($dbname) or die( "Unable to select database"); //Adds by Google: if you want to use it, delete the outcomments /* $googleads = '<p>
<script type="text/javascript"><!-- google_ad_client = "pub-123myid"; google_ad_width = 336; google_ad_height = 280; google_ad_format = "336x280_as"; google_ad_type = "text_image"; google_ad_channel = ""; google_color_border = "EFEFEF"; google_color_bg = "EFEFEF"; google_color_link = "496D9F"; google_color_text = "000000"; google_color_url = "008000"; google_ui_features = "rc:0"; //--> </script> <script type="text/javascript" src="http://pagead2.googlesyndication.com/pagead/show_ads.js"> </script>
</p>'; */ $googleads = ''; // outcomment this line if you want to use Googleads. I prefer to have the Googleads in the footer of the News. $news_queue = '0'; // 1 moves all the news into the queue, 0 moves them directly into the news $globreplacestract = '1'; // I had difficulties with accents in German, French and Italian. I don't know if it's utf-8, but it works fine for me with this general replacings. $globreplacestrfrom = array(''', 'à', 'á', 'â', 'æ', 'ç', 'è', 'é', 'ê', 'ë', 'À', 'Á', 'Â', 'Æ', 'Ç', 'È', 'É', 'Ê', 'Ë'); $globreplacestrto = array('\'', 'à', 'á', 'â', 'æ', 'ç', 'è', 'é', 'ê', 'ë', 'À', 'Á', 'Â', 'Æ', 'Ç', 'È', 'É', 'Ê', 'Ë'); ////////////////////// // Single RSS feeds // ////////////////////// // 1st Feed $feedid = '123'; //set feed_id $feedlanguage = 'english'; //set feed_language in DF tables $feedtopic = '123'; //set topic id in DF tables $feedcategory = '123'; //set category id in DF tables $sourcestract = '1'; //1 turns the "source" line on, 0 turns it off. $sourcestrstart = 'News by: '; //initial string of the "source" line. $sourcestrlink = 'http://www.LinkToTheFeedWebsite.com'; //link to the "source". $sourcestrname = 'Feedname'; //name of the "source" link. $sourcestrend = '<br />'; //end of the source string $authorstract = '1'; //1 turns the "author" line on, 0 turns it off. $authorstrstart = 'Source:'; // initial string of the "author" line. $authorstrend = '<br /><br />'; //end of "author" string $getstorystract = '1'; //1 turns the "Get the full story" line on, 0 turns it off. $getstorystrstart = 'Get the full text '; //initial string of the "Get the full story" line. $getstorystrend = 'here'; //linked final string of the "Get the full story" line. $truncateact = '1'; //1 truncate the content, 0 turns it off. $truncatewhen = '450'; //truncate content if longer then this value $truncateto = '400'; //truncate the length of the content to this value $replacestract = '1'; //1 turns the replacement of strings on $replacestrfrom = array('replace1', 'replace2', 'replace3', 'replace4'); //expand the list of strings to replace from $replacestrto = array('to1', 'to2', 'to3', 'to4'); //expand the list of strings to which you like to replace $replregexpact = '1'; //1 turns the replacement of POSIX regular expressions on $replregexparray = array( array( 'pattern' => '(pattern 1)', //POSIX pattern 1 'replacement' => 'replaceto1' //POSIX replacement 1 ), array( 'pattern' => '(pattern 2a)+(.)*(pattern 2b)', //POSIX pattern 2 'replacement' => 'replaceto2' //POSIX replacement 2 ), array( 'pattern' => '(pattern 3)', //POSIX pattern 3 'replacement' => 'replaceto3' //POSIX replacement 3 ), array( 'pattern' => '(pattern 4)', //POSIX pattern 4 'replacement' => 'replaceto4' //POSIX replacement 4 ) ); //-> expand this array to add more POSIX patterns include 'rss_update.inc'; // Amazon US $feedid = '29'; $feedlanguage = 'english'; $feedtopic = '8'; $feedcategory = '31'; $sourcestract = '1'; $sourcestrstart = 'News by: '; $sourcestrlink = 'http://www.amazon.com/exec/obidos/redirect?tag=YourAmazonID&path=subst/home/home.html'; $sourcestrname = 'Amazon "Amazon Seachrchterm"'; $sourcestrend = '<br /><br />'; $authorstract = '1'; $authorstrstart = 'Author(s):'; $authorstrend = '<br />'; $getstorystract = '1'; $getstorystrstart = 'Get the full text '; $getstorystrend = 'here'; $truncateact = '1'; $truncatewhen = '450'; $truncateto = '400'; $replacestract = '1'; $replacestrfrom = array('<br /><br /><br /><br />', '<br /><br /> <br /><br />'); $replacestrto = array('<br /><br />', '<br /><br />'); $replregexpact = '0'; $replregexparray = array( array( 'pattern' => '', 'replacement' => '' ), array( 'pattern' => '', 'replacement' => '' ), array( 'pattern' => '', 'replacement' => '' ) ); include 'rss_update.inc'; //close MySQL connection mysql_close() ///////////////////////////// //// STOP PRIVATE SCRIPT //// ///////////////////////////// ?>




rss_update.inc
<?php //get rssitems $query = "select * from rssitems where feed_id = '$feedid' and news_submitted = '0'"; $result = mysql_query($query); while ($row = mysql_fetch_object($result)) { //set variables $item_id = $row->item_id; $feed_id = $row->feed_id; $checksum = $row->checksum; $title = $row->title; $url = $row->url; $author = $row->author; $posted = $row->posted; $updated = $row->updated; $descr = $row->descr; $imageurl = $row->imageurl; $imagehight = $row->imagehight; $imagewidth = $row->imagewidth; $imagelink = $row->imagelink; $imagetitle = $row->imagetitle; $guid = $row->guid; $new = $row->new; $news_submitted = $row->news_submitted; $title = stripslashes($title); $author = stripslashes($author); $descr = stripslashes($descr); $imagetitle = stripslashes($imagetitle); //trim whitespace at the start and at the end of the entry $title = trim($title); $author = trim($author); $descr = trim($descr); $imagetitle = trim($imagetitle); //update title if ($globreplacestract == 1) { $title = str_replace($globreplacestrfrom, $globreplacestrto, $title); } //update descr //replace strings if ($globreplacestract == 1) { $descr = str_replace($globreplacestrfrom, $globreplacestrto, $descr); } if ($replacestract == 1) { $descr = str_replace($replacestrfrom, $replacestrto, $descr); } //replace regular expressions if ($replregexpact == 1) { foreach ($replregexparray as $replrow) { $descr = eregi_replace($replrow['pattern'], $replrow['replacement'], $descr); $descr = eregi_replace("($author)", '', $descr); } } //truncate descr if ($truncateact == 1) { if (strlen ($descr) > $truncatewhen) { $descr = substr($descr, 0, $truncateto) . '...'; } } //add image to descr if (!empty($imageurl)) { $imageurl = '<a href="' . $url . '" target="_blank" rel="nofollow"><img src="' . $imageurl . '" border="0"></a><br />'; } //add infos to descr if ($sourcestract == 1) { $sourcestr = '<b>' . $sourcestrstart . '</b><a href="' . $sourcestrlink . '" target="_blank" rel="nofollow"><i>' . $sourcestrname . '</i></a>' . $sourcestrend; } else { $sourcestr = ''; } if ($authorstract == 1) { $authorstr = '<b>' . $authorstrstart . '</b> <i>' . $author . '</i>' . $authorstrend; } else { $authorstr = ''; } if ($getstorystract == 1) { $getstorystr = $getstorystrstart . '<a href="' . $url . '"target="_blank" rel="nofollow">' . $getstorystrend . '</a>!<br /><br />'; } else { $getstorystr = ''; } $story = $sourcestr . $imageurl . $authorstr . $descr; $storyext = $getstorystr . $googleads; //replace strings again if ($replacestract == 1) { $story = str_replace($replacestrfrom, $replacestrto, $story); } //replace regular expressions again /* if ($replregexpact == 1) { foreach ($replregexparray as $replrow) { $story = eregi_replace($replrow['pattern'], $replrow['replacement'], $story); } } */ //show the updated content echo $title . '<br /><br />'; echo $story . '<br /><br />'; echo $storyext . '<br /><br /><br /><br />'; //addslashes $title = addslashes($title); $author = addslashes($author); $story = addslashes($story); $storyext = addslashes($storyext); $imagetitle = addslashes($imagetitle); // avoid dupplicates $title_new = $title; $story_new = $story; $title_new = eregi_replace("[[:space:]](-)[[:space:]](.)*", '', $title_new); //Google pattern $title_new = eregi_replace("[[:space:]](\()(.)*(\))$", '', $title_new); //Yahoo! pattern $query_old = "SELECT * FROM cms_stories WHERE title LIKE '%$title_new%' or hometext = '$story_new'"; $result_old = mysql_query($query_old); while ($row_old = mysql_fetch_object($result_old)) { $title_old = $row_old->title; $hometext_old = $row_old->hometext; $title_old = stripslashes($title_old); $hometext_old = stripslashes($hometext_old); $title_old = trim($title_old); $hometext_old = trim($hometext_old); $title_old = addslashes($title_old); $hometext_old = addslashes($hometext_old); $title_old = eregi_replace("[[:space:]](-)[[:space:]](.)*", '', $title_old); //Google pattern $title_old = eregi_replace("[[:space:]](\()(.)*(\))$", '', $title_old); //Yahoo! pattern $title_old = strtolower($title_old); // change string to little letters $title_old = ucwords($title_old); // first letters of words in string are big letters $title_old = eregi_replace("[^a-zA-Z0-9]", '', $title_old); // remove everything else then a letter or a number } mysql_free_result($result_old); $title_new = strtolower($title_new); // change string to little letters $title_new = ucwords($title_new); // first letters of words in string are big letters $title_new = eregi_replace("[^a-zA-Z0-9]", '', $title_new); // remove everything which is not a letter or a number if ($title_old == $title_new or $hometext_old == $story_new) { echo 'Duplicate "' . $title_new . '" has not been submitted to the News!<br />'; } else { //puts item to news queue or directly into the news if ($news_queue == 1) { $query2 = "INSERT INTO cms_queue (uid, uname, subject, story, storyext, timestamp, topic, alanguage) VALUES ('240','News', '$title', '$story', '$storyext', '$updated', '$feedtopic', '$feedlanguage')"; mysql_query($query2); } else { $query2 = "INSERT INTO cms_stories (informant, title, hometext, bodytext, time, topic, catid, alanguage, ihome, acomm) VALUES ('News', '$title', '$story', '$storyext', '$updated', '$feedtopic', '$feedcategory', '$feedlanguage', '1', '1')"; mysql_query($query2); } } echo 'old:' . $title_old . '<br />'; echo 'new:' . $title_new . '<br />'; //set news_submitted to yes $query3 = "UPDATE rssitems SET news_submitted = '1' WHERE item_id = '$item_id'"; mysql_query($query3); } mysql_free_result($result);

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Linux/Apache/MySQL 5.0.45/PHP 5.2.2/DF 9.2.1

All times are UTC